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* Presented is the executive evaluation report 
Proj6ct^City Science fPC'S) - The PCS, a program conducted by 
University and 'fhrided by NSF, sets its major ^al tHe xmp 
of junior higfi school science instruction in tff5 inner-city 
.environment. This report e consist§ of fiv^ sections* (1) over 
the project; (2) operation of the project"; (3) critical «sse 
the overall project; |«) general conclusions; and (5) 
recommendations.- The first section coversj funding and frojec 
and evaluation' procedures. The second, section covers staf fin 
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university /school relationships, and effectins change. ^The^f 
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planning, and legacy of %he project. The fifth Section offer 
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of the . 
New lork 
rovement - 

viev of 
ssment - 

t intent, 
g and 
The third 

gy# 

ourth 
tatus, 
s tvo 



» t < 

* Reproductions supplied by EDRS are the best tnat can be made * 

* from the original document. * 
***********************************^ 



ERLC 





♦ 



U S DEPARTMENT OF EDUCATION 

NATIONAL INSTHUTE OF EDUCATION 

EDUCATIONAL RESOURCES INFORMATION 
xa CENTER^ERICI 1 / 

X This document has' been reproduced as 

received from th* person or organization 

originating it 

Minor changes have been made to infiprove 
reproduction quality 

• PointbOt view uf opinions st^t'd in H11& (jocu 
rnetit do not necessarily represent official NIE 
position or poliLV ^ 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 

Linda . 7 T.M\sc\ri 



vD i 

o. : 
o j 

CM j 

a 

UJ 



PROJECT CITY 
SCIENCE 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) ** 



is 




591 



V 



" Siiil^lftl^.TQ THE 
r^TION^; SCIENCE FOCINDATJON 




/ 



FINAL REPORT: EXECUTIVE SUMMARY 
«- PROJECT CITY SCIENCE 
1979 



Report -submit fed>y 
Queens College to the 
.National Science Foundation 
August 31, 1979 



r 



4 Evaluation Staff - 

"Dr. Paul Longo, Evaluation Director 

f . 

Dr. Philip White 
* Dr. Daniel Brovey 
vDr. . Louis Hofmann 
, Dr. Alan Simon 
*Dr. Richard White \ 



Advisory Panel 
thr. J. Myron Atkin 
Dr. Jerome Notkin 
Dr. Vincent Reed 



The material irj this report is based. upon work supported by the National 
Science "Foundation undef L Colfvfcract No.. SER 77-26936. Any opinions , findings, 
and conclusions or r^cja^^atidfrra ^xf^cessecj in this publication are those 
of the author and do not rfeces^sarily reflect thr views of the National 
Science Foi/ndatipn. 



r • ■ TABLE OF CONTENTS 



OVERVIEW OF THE PROJECT 
Funding Intent 



Project Intent -. . { . 4' 

J 

Summary of Original Proposal 9 

. Evaluation Plan and Procedures 17 

II OPERATION 'OF THE PROJECT 

Staffing and Organization . 20 

Modifications • . .24 

Support for Implementation 4 .. j 26 

V \ 

Ill CRITICAL ASSESSMENT: THE OVERALL PROJECT ' » * 

Funding Implications ✓ * 29 

Instructional Methodology t 31 

University-School^Relationships . ■ ■ . * 33 

Effecting Change. B . . # . ■ .37 

IV GENERAL CONCLUSIONS 

/ % 

Vision ;.../...' 41 

Scope ■ : . ■ .42 

Status * ; *. ./ * 44 

/ 

Planning ./ 46 

Legacy 7 50 

port Format * * * . ..53 

V RECOMMENDATIONS 

y . * * \ 

Preservice Program m m . v 55 

Model District Program ^ \.56 

Dissemination Program ' . . 73"T. . . : r . t . . . / . 58 

Research Program 1 : . * . . * : ; . . #p9 

Policy Recommendations ? 1 61 



^ J 



t 

erIc . i 



• 5. 



ACKMOWLEaXJEI^ITS 

The evaluation team would like to extend its sincere appreciation to the 

Project City Science staff. Their cooperation throughout the coiarse of the / 

♦ 

evaluation has been extremely helpful. Under adverse conditions, and a heavy 
teaching schedule, the staff remained personable and responsive to the needs 
of the evaluators. Throughout the course of the evaluation, they suffered 
our presence with considerable patience and ^race, retaining their candor - * 
and a balanced, perspective even when what the evaluators ha& to say w*s not 
completely favorable. The <staf f was thoroughly ^cooperative and ve are thankful 

9 

for the courtesy that they extended to us. 

j 

we would also like to thank the Project Director, Dr. Fletcher wutson, 
'whose competence and insist impressed us. His grasp or operational aspects 
of the Project was excellent, and his understanding of the Project staff's 
capabilities wus siuite detailed. He lent a measure of leadership and 
administrative aoixi-cy which whs extremely helpful to the Project during 
a difficult time. As evaluators, we appreciated his presence and the quality 
of his efforts on behalf of the Project. / 

we coula not close without mentioning the contribution of the_ .associate 
Project Director, Dr. James Connor, whose willingness to ^ive of his time 
seeded of immeasurable importance to the Project. Due to changes in leadership, 
his role at key junctures was critical, and his contribution- throughout was . , 
significant.. Much was »sked of ' him,,, and he responded well, ffl-s efrorts were 

♦ 

respected "by both the Project and evaluation staffs. 



•7 



5 



X 



a* was the case with the Project leadership, we round the rest of the 
staff- to be con enial,-ablc,. and 'consistently courteous. We -are most thankful 
for the way they received us. The :.iaJor regret the evaluators had was the 
reeling that' we may have been less helpful to Project stafr in specific, - 
individual ways than we could have oeen. Perhaps that was a fault in our 
definition or the role. Dr. Watson has> offered a recommendation that is not 
unreasonable and vhich ye £eel merits serious consideration by future 

evaluative groups: \^ 

I vish to make a suj;,:estion vhicin* could-reduce the J 
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I, OVERVIEW OF THE PROJECT 

» * 

A, Funding Intent . 

During the "early part of this decade, the. policy of the National Science 

Foundation (NSF) had been expanding irom an earlier emphasis on the develp^- 

ment of teaching talent to include attempts to support large-scale efforts 
\ 

aimed. at influencing broad educational networks. This latter evolution can 

• . ■ / - 

be traced in the Foundation's development > of support for Comprehensive Programs, 
and, later, the Systems Awards, both of which stressed concern with issues that 
were more intricately interwoven within tire deeper structures of education. 
The problems addressed were broader and solutions were aimed at the level of 
fully developed systems r # ather than at discreet individuals working wxthin # 
them. 

The Foundation was also re-examining at this time the issues of impact 
and residue,* i,e,, how the projects supported were influencing the education 
of'Vtudents, and which influences remained operative when fundirfg was with- 
drawn. The re-examination led to the support of long-term efforts in which 

the Foundation committed itself to multiple year funding. Such an approach 

• * 

allowed recipients to design programs that were longitudinal in nature, 
and whose assessment could contribute in important ways^ to ouf knowledge of 
particular aspects of the educational effort. This approach represented a 
significant departure from the earlier practice of supporting non-product 
oriented interventions on a short term-basis, which had been the predominant 

mode, ' 

« 

In assessing these more broadly conceived, longrterm proposals, 'support 



*An' emphasis which manifested itself mainly in the form of summer .and academic 
year institutes that provided training for in-service teachers. 



was given to projects that focused on a specific meed or problem within the 
larger educational system, and which offered a design for coordinating the 
efforts of relevant agencies and institutions concerned with the identified / 
problem. In pursuing this policy, the Foundation yas seeking to allow, the % 
directors of sudh projects V greater amount of discretion and flexibility. It 
was further hoped that these long-term cimmitments would result in 'decisions 

7 - - t 

by local funding agencies to offer' financial backing for project functions 

V 

whose usefulness had been'demonstrated during the time that.NSF had supported 

r * 

them. ^ ' 

Project City Science (PCS) represented one such effort.* PCS appeared^ 
to be an* important; zkd in many ways necessary venture on the part of NSF, 
tx> support a broadly conceived innovative educational venture in an urban 
environment. -The concept was, for a number of reasons, an idea whose time 
had come. The nation was increasingly concerned with the social consequences 
of urban decay and dislocation. The writers of the original proposal put 
forward a sound analysis of the plight of the jfchbol's in the inner cities of 
the nation. The compelling need to learn more about the environments in 
which increasing numbers of our young "people were being educated was clearly 
and skillfully developed. The Project had the- foods that the Foundation was 
seeking and offered a meatus of coordinating the efforts of a number of agencies 
concerned with the problem. 

Beyond specifying a # particillar environment (the inner city), the pro- 
posers of the Project went on to identify a specific level (the junior >iigh 
school), and function (science) within the educational 'structure that clearly 



*The Project was proposed and conducted by members of the Department of Science 
at New Yor\c University. " 4 
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•required greater attention than they had heretofore been given. " , 

The junior high school has long been an anomaly within the public school 

structure. Despite its status, all too little attention has been paid to the 

f 

unique set of problems faced- by educations at that level^ The developers of . 
.PCS addressed these concerns with skill and imagination. |They presented a' 
perspective that provided a broad .view of the problem, and recognized the 
need to specially train pre-service teachers for this environment. The 

«. proposal indicated an intent to offer additional^ aid to those currently in ^ 

\ V i, 

service, and to develop a school-university support system that would make 

J 

- available the best that each institution had to offer. It- wa S> a well con-^ 
ceiv.ed design, ambitious in scope, although perhaps overly optimistic in its 
.stated and implied expectations. , The design clearly encompassed major problems 
. faced by eaucators at 'junior' high school level, and offered the promise of 
dealing with them through a coordinated, inter-institutional effort, conu- 
' prising research, teacher training, and implementation of innovative ideas. 

Project City Science became one of several large scale, long-term programs 
•supported by NSF. i In supporting such projects, the Foundation appears to have 
' had several sets of expectations—some more explicit than others. Firsts of 
purse, was the hope of fulfilling the particular purposes for which each 
project had" been designed. Second, and perhaps 'of lesser concern to the 
internal project' management,' was the Foundation's hope that the programs sup- . 
ported might not only accomplish their purposes', but do so in objectively 
measurable ways. A further hope was that a residual effect of such aid would 

• be the willingness of local. funding sources to assume tbjf support of certain 
worthwhile project functions. 

To aid in accomplishing. these purposes, each of. the^ pro jects^ including 

• PCS, was provided with an internal evaluation capability. In the case of 
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Project City Science, an external evaluation mechanism, was also provided. 

The ultimate purpose of. the evaluation was to provide an outside perspective 

** * 

on what the Project had accomplished. This included an assessment of 'its out- 

\ 

comes during the funded phage, and an estimate of its continuing influence 

i and prospects for the future, A$ stated in the Foundation's original -charge 

to the evaluators: 

(The) third party evaluation is requested to provide a 
summative look at the funded period of the project from an \> 
external perspective. The evaluation should include a 
✓portrayal of the project indicating its strengths and weak- 
nesses as determined by both the examination of existing 
data and the collection of new data. In addition, the study 
should provide insight on possible future direction^ for the 
project during the remaining ten years and aid NSF in making 
future policy decisions regarding projects of this type, 1 

B. Project^ Intent % * 

As noted earlier, the submission of' the PCS proposal coincided with, and 
was re^fofr^ive to; * 

a stated NSF interest (in) 4 experimenting) with more flexible, 
. raor^ sharply focused and* more fully coordinated approaches to 
' ' staff development and support activities than was possible 

in other toSF programs, such as teacher institutes, 2 
• - * 

The* PCS proposal was based \>n a conscious decision by the leadership in the 

science education department at NYU to. identify an area of need that was large 

enough to require their^attention, and yet small enough to respond to efforts 

at improvements. It was decided that, science at the junior high school level 



me 



t these conditions. That it was an important area of need, few would disput 



The NYU staff also felt that as a project focus, it represented a problem of - 
manageable proportions — onethatwas in their opinion sufficiently well defined 



*The PCS staff has noted that a. fuller external evaluation ,was not provided un 
the fourph year of the Project's existence, which in their view was rather lat 
to be of suilicienr help. , 

JO 
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to admit of possible solutions. That view, was most clearly expressed by the 
Project staff in its response to an evaluation of Phase J (the first two 

years^of the PCS effort ' * 

< Wfe^do not regard the gigantic .City as our abject of ^ 

4 interestX^We are interested in only a small but, very, < 

. important p^ft of, the Cdfcy, namely the teachers and students 
in the intermediate science classrooms, aqd we do not regard 
these teacher's and students as resistant foes to.be overcome. 
The teachers are individuals who can change and become more 
effective* if they are given help in- clarifying goals, and 
provided with new insights concerning their students and 
their situations.^ 



As is clear from the response cited. abtfve, the Project staff had a 

reasonably clear picture of its major intent. It was one of. revitalizing 

science teaching within t-he junior high schools of New -York City, In the 

process, the staff hoped to discover and document.' some things that would be . 

, useful to educators* in other c urban areas,^ The-<mnjediate purpose was "to . f 

'develop two model districts within the New York Cityrsystera that would reflect 

■■4 - ' 

"the highest possible level of intermediate science teaching," Amore long 
range intent was to gradually fill a significant portion of the Cit;y. s junior 
high school science teaching positions with PCS graduates who had been ^ 
specially trained to work in the urban environment, * Indeed, M:he Project^ 
leadership entertained the hope that; - * t 

If Project City-Science succeeds, and if it is duplicated' in 

^ other cities,' in ten years we could replace up to 40 percent 

(of junior high science staffs) with a cadre ofc science ' . 

teachers 'rained for the job,^ 
« 

' In brief,' these two statements epitomize the nature of the long and short range, 
. purposes to which the Project had committed itself.. The Project sta/f believed 
chat the junior high occupied'a position of unique. importance in-shapihg ^ 
students'' decisions to continue their education *a science,' since most of the 
)es offered 'in the high school are elective, rither than mandatory. 

.u 



cop* ^e: 
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* In working toward the accomplishment of the goals described above, the 
staff anticipated the achievement of^foiir objectives: 

1. The development of the two model school districts. \ ■ 
*» v 2. A unique pre-service training program at NYU. ^ ^ 

3. A research and evaluation institute. ' • m 

4. A strategy model for institutionalizing change. . ^ 

In explaining what PCS was attempting* to accomplish, it is important to 

• point out the difficulties the Project faced as it prepared to implement oitSA 
design. The staff had already chosen perhaps the most drff-icult educational^ 

r 

environment to work in — that of the inner city. The difficulty was furthers 
compounded by the fact? that the Project was conducted in the most populous, 
and perhaps the most varied and cQmplex, of ttje nation f 's cities.. Beyond this, 
the staf/ had'decitied to work at a level within* tfte school system, which 
while surely in need of aid, has been an 'enigma to many educators; the educa- 

■ -'' \ \ " ' % -n' 

* tion of adolescents is still the most ctonanding v *and challenging task facing 

' ' • 7 • " • ' ' ' x 

tShe sthools. While acceptance of that .challenge gave the Project <mtted 
importance, it al$b posed a serious test/ 'of the skilly and resourcefulness of r 
the staff selected to implement the objectives. The vision and foresight of 
those who would hirfe, the staff were to be puts? to the" test, for the scopfe of 

- ■ • • *•■''. * ■ 

the task assumed was audacious. * 7 > 

*' . ' 

Within this complex geographical setting and demanding instructional level; 

i\ * * * 

two school districts were selected that themselves offered numerous challenges. 

* 

Whether by designer through oversight/ fetf concessions were made to the mount- 
• *• • 

ing list ot difficulties "with which the Project would have to deal. The 

apparent intenj;" was &o face, as completely as possible; the full set of burderife 

encountered* 1)y teachers wording in this environment. • * - 

; ' ~* ' ^ 

%*TKis later evolved, or was replaced by, a. program of dissemination. 
t * 

12 " x • . ; 
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During the second year of its existence, the Project was faced with 

a fiscal crisis in New York City that threatened to destroy much of what, had 

previously been dcQomplished, and' which- thoroughly up'set all plans for the 

future. There we^e massive lay-offs of instructional staff, indiscriminate 

reduction of support services, and wholesale reassignment of teachers with 

seniority to positions for which they were neither licensed- nor trained. The 

.Project staff/which had earlier attracted a number of younger innovative 

teachers £o participate in their activities, stRldenly found that most of 

those with tyhom they had been working were no longer employed.^ Compounding 
* 

the problem was the fact tfrSt" many of those who replaced them had little 
'experience and even less trainKg 'in teaching science. Much of what had been 
initiated needed to b6 reinstituted and most of what had been planned re- 
designed. It must be .understood that the Project faced more* than a simple 
change in personnel in the schools. The entire* climate was altered. -Teacher 
morale had been dealt a serious blow, and the PCS afetf^was working in an 
environment which was far less hospitable to the concept of school staff 
-committing' extra time and effort to the improvement of teaching. Even, where 
attitude^ remained more wholesome, energies were consumed by the more corapel- 

ling demands of maintaining » badly shaken system of education. Few in such 

' ' / * ' 

a climate were prepared tOvturn their , attention to the implementation of 

•\ - • ' / 

• innovative instructional techniques. 

* « * 

1 -Finally, to the-problems facing New York City must be added the economic 

difficulties which, flew 'Yox;k University itself was undergoing. The Project, 

designed during a.'grpwth period, was actually funded during a time when the 

'; ^ , 
.University was experiencing some fiscal problems of its own. The result was 

a reduction of th^/jftotal staff, thereby placing great. restrictions upon the 

* / 1 

availability of supporting services. PCS, predicated upon a university-wide 
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support System; now found it more difficult to marshal the typje of help once 

hoped for. The problem was exacerbated by the fact that the Department of 

» 

Science Education was also experiencing shrinking enrollments. The crisis in ' 

I * 
New York City made the prospect of obtaining a teachiifg position appear poor, 

and so student application^. fell accordingly. Mounting economic restrictions. 

•seeded to settle upon the Project from every direction. 

It is difficult for evaluators to render a judgment as to how damaging 

such a series of crises were to a fledgling project struggling to establish 

itself. It certainty seems reasonable to assume that they represented a 

seriobs blow to the enthusiasm and hopes of the staff. Providing leadership 

during such a time is difficult. The maintenance of the Project vision and of 

staff morale must have been particularly hard. There was considerable staff 

turnover during the^ early years of the Project. Some of this was planned,* 

but not all. The loss of so many staff members, and particularly "those who 

were mission-oriented, could well have taken the edge off their efforts. 




What is clear is that Project City Science was faced with a^d^fficult set of 
additional burdens early in its history. The extent to wfiich that inhibited 
- Project achievement will never fully be known. 

What has been described above is not offered as an apologia for the 
Project. The evaluators do not seek to imply that one was needed, nor would 
we attempt it. While there were problems, there were rich opportunities as 
well. What we have tried to do is to describe objectively, and in fairness 
V to tljglPCS staff, the historical circumstances in which the Project found 

itself as it attempted to implement its design*. >te note that these events 



That this was never completely true fc?r positions in science and mathematics 
was not clearly understood by many teaching candidates, although it was to 
become more readily apparent with each passing year. 

o ' ' ' 14 - 
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took place prior to the evaluation period for which we are specifically 
responsible.* A number of the circumstances described were ; f ar less pressing 
during the time of 'the current evaluation, but we would not interpret our 
charge so narrowly as to p*fclude'the possibility of their exerting a con- 
tinuing influence. In pointing to them, we hope to provide a backdrop 
against which £he Project 1 s 'continuing activity might be examined and better 
understood. Where the constraint upon achievement is attributable to the m 
general environment, it should be noted, as must be the failure of the 
Project staff to fully seize or capitalize upon the opportunities that 
their unique position afforded them. 

C. Summary of Original Proposal 
1. Funding 

Project" City Science was initially funded' by NSF for a 34-month period, 
beginning in May, 1974, and terminating on March 31, 1977, though a subsequent 
proposa^indicated that: 

the intention was for most of the funds .to be. expended over a 
two-year span ending August 31, 1976. The grant was made to 
underwrite the initial phase of what hopefully would become a 
three-phase, fifteen-year plan to greatly improve science 
teaching and learning. in the large cities of America. 

In accordance with this plan, ^second proposal was submitted by N.Y.U. 

requesting.. and receiving funding for an additional three years of operation 

that would- constiture" the second phase of the original design. , Such funds were 

' to insure eoptinuatio« of .the project from September of 1976 through the summer 

of 1979. Thus, the entire duration of outside funding was to extend over a 

total of five years, though there appeared to be some overlap in the funding 



, *This evaluation covers the secjjnd^phase of the Project's operation, 1976-79. 
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protided fo* Phase I (1974-77) and Phase II (1976-*T$«&) 

Phase III of Project City Science was expected v to continue for another 
ten years beyond this initial five-year funding period. ¥h±3^La$t phase was 

to be entirely self-suppotfcing. 7 The PCS staff would use th< "fuijds initially 

•i i 
provided to create the structure upon, which the, continuing 0{ eration of the 

program would bk built. Trie intent was to make the PCS model' a part o^ the 

permanent structurksa^LNew Yor^ University as well as i^ro develop a strong j • 

base of support in the New York City schools and surrounding colleges and ▼ 

universities. The proposal* noted^hat at the end of these years of funding, 

Project functions and activities will be self-sustaining. 

2. Project Intent 

• - 

Project City Science represents an attempt to examine and deal with the 

problems of education in the urban seating. The proposal notes that while 

nearly seven out of every ten school children in the United States reside in 

metropolitan areas,. the schools they attend too 1 of ten reflect the limitations 

of the urban environment: persistent overcrowding, a rapid flux of ethnic 

population, a steadily increasing proporti<fn of the very poor to be served, 

deteriorating physical facilities^ and a shrinking 'financial base. In the 

<S 

view oT the proposal writers, one consequence of these limitations is that the 
quality of education in American cities has declined sharply and there is an 

urgent need to develop means of addressing the problems that have resulted. 

— s - : 

PCS Was designed to deal specifically with one dimension of that problem, 
science education, at a particular instructional level, the junior high school. 
The proposal states the major intent of the project as follows: 



Unless otherwise specified, th*e proposal referred to wift. be the full proposal 
dated 12/1/75, which was initially 'submitted requesting fund^fig for-Bhase. II of 
the Project. * 

: 16 



1) to put together a cooperative effort in New York City 
involving teachers' in the city schools, the teachers 1 union, 
administrators at achool^ district, city and state levels, 
community organizations, p*£of essional associations, and 
several universities within the city, a coalition that can 
bring about oyejr a fifteen year period a dramatic improvement 
in the teaching and learning of science in the intermediate 
schools (grades 6 through 9); 

2) to do this in sucjv a way that the reform process becomes 
continuous and * institutionalized; and 

3) at the same time, to generate and' disseminate knowledge 
about adolescents; the learning of science in the inner city^ 
situation, and the process of improving science instruction. 

In $ later section of the proposal, what is referred to as the central 

purpose of the project is restated^r^m the original (1974) proposal: 

to help bring about a major, lasting and self-perpetuating 
improvement', principally in New Yojrk City, in the teaching 
of science in the middle grades between elementary and high 
• school 

While the rationale for placing primary emphasis on science rather than other 

subjects, such as reading or mathematics, is not clearly stated, it is evident 

that the proposers of/fcs feel that science is an area in which instruction is 

particularly ineffective. It is - * noted that, 

science teaching at the middle school level in New York City 
and many other cities can only be regarded, on* tho whole, as 
gravely inadequate. . . (Further), science education in the 
city elementary schools 'remains woefully weak; when not absent 
altogether. H 

% 

Having concluded that, "improving elementary school science in the cities 

12 

seems -to be an intractible problem of massive proportions Project staff 
* 

apparently decided that the middle school (i.e., grades 6-9) should become the 

\ 

logical focus, of their efforts. The reasons offered °for this appear to be 
three-fold. First, a large fraction of inner city youth do not go 6n to attend 
high school, and so efforts made at a later stage would be too late. Sefcond, 



by the time students reach high school, a deep antipathy toward the study of m 
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science has already developed, and so they will .usually not choose to take . 
courses in science; third, even though many educators agree <the junior high 
school years may be critical for students, very little emphasis has been 
placed on developing procedures that improve instruction or modernize curricji- 
^lum at this level—particularly in .science. 

The proposal goes on to clearly emphasize its junior high school, focus. 



For many city youngsters, junior high school provides the 
* only formal instruction in science they receive in their 
lives!. . . it constitutes quantatively the most science 
they will formally encounter. H 

Project emphasis was not solely upon the direct improvement of science 
instruction in the school, but upon the development of a model program for 
training junior high school science teachers as well. The intent was to both 
* provide science teachers for the New York City middle schools, and to develop 
a training model with widespread potential. The then Project Director, inter- 
viewed 'for an article about PCS*> indicated what the program's major concerns 

were: ; 

First, we're doing inservice training of teachers who are 
already in the schools. Second, we're designing a training 
program for the whoie next generation of junior high school 
teachers. Tnird, we're working, to analyze instructional 
problems and devise system-wide solutions.'* . • . Over the long 
run, (the Director) can envision Project City Science helping - 
.to effect a new kind of science teaching. ... If Project City 
Sconce succeeds, and if it is duplicated '-in other cities, in 
ten years we could replace up to 40 percent with a cadre of 
.science teachers trained for the job. . ... What we want to 
develop is- a design that can be used in city schools through- 
out the countuy, something that can be adopted. quickly by 
other universities and other school districts. 14 

3. Project Goals 

* ' ' " f 

Since the* funding provided for Phase II •of the project was substantially 

less than that originally requested, a revised proposal was submitted to NSF 

•» by New York University restating what was to be accomplished. The goals of 

j 18" " 
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the Project had. changed very little, though the revised proposal notes that, 

At the level of funding (provided). . . it will not be 
possible to accomplish (theft) as rapidlyas originally 
proposed or \fith the same probability of success. 15 

Nonetheless, the proposal clearly states that: 

The main purpose of Project City Science remains unchanged: 
Go improve intermediate and junior high school teaching in 
New York City and to lfearn something in the process that 
will be useful to colleagues in other universities and in« 
other urban areas ... (Program cutbacks would be) under- 
taken using* three criteria: ^ — > 

1) The Project's chief characteristics must be preserved. > 
• These include utilizing a cooperative and functionally 

comprehensive approach, keeping the school district* as 
the chief -unit of attention, being knowledge-generating 
and making and keeping long-term commitments. These 
features were to be regarded as more crucial tharf 
extensivenSss and magnitude. ^ 

2) Those activities most likely to lend* themselves to 
institutionalization should be favored. To insure 

continuing reform, this must -be sought in the uni- ^ ^ 
versity, .school, and community setting. 

* * 

3) Whatever is to be undertaken must contribute po the 
development of a concrete, describable, visable entity 
or product that has dissemination capabilities. 

While the Phase I aspect of 'the project that was initially funded dealt 

with 16 separate areas, the revised proposal submitted for Phase II functionally 

reduced these to four areas in which a major effort would be concentrated: 

At the level of funding now, available, the project will work 
toward the achievement of four definite "product s." These are: 

1) two model districts;" 

2) a unique pre- service program; * ; 

3) a research and evaluation institute; ^ 

4) a strategy model for change arid institutionalization. 1 ^ 

The development of these four "products," then, is set forth as the majot 
goal of the current phase of the program with which this evaluation is concerned. 
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'The report will attest to clarify the anticipated outcomes of each of the 
four major aspects of the PCS program and comments on the effectiveness of 

4 * * ' 

the effort the project staff has mounted to attain them. In analyzing 

these Efforts, it is useful to keep in mind the major problems that the pro-' 

posers of Project City Science felt PCS was creatad^t^address, > 

Assuming, then, that there is an especially urgent need to 
improve science instruction during the transition years, 
wh,at are the particular problems that must be solved or at 
least ameliorated? The 1974 proposal explicitly claims, 
and Project experience has subsequently supported, that 
- three major problems exist : 

1) The failure of teacher training , both preservice and 
inser1ft.ce, to prepare science teachers to deal effec- 
tively with t&e early adolescent child in the inner- 
city situation. f 

2) A continuing reliance on science programs that do not 
* reflect sufficiently what has been learned in > the last 

decade or so about science curricula and new approaches 
to teaching science. 

3) A scarcity of systematic knowledge about the age group 
and about what conditions and techniques best promote an 
interest in a 'learning of science at that age and *h 
inner-city circumstances. 

Implicit in the proposal and accentuated by Project 
experience "is a fourth problem: The failure on all sides" 
to identify, organize, and hrinft to bear in a coordina ted . 
way the not inconsiderable material and human resources of • „ 
the state; city, district schools, universities, <and 
community at large. Related to this is the problem of 
establishing a se,lf-sustaining system for continuing reform 
rather than merely , instituting tljis or that improvement, 
regardless of hbw alluring a given reform seems to be in 
the short run, or however much desired by one or the other * 
agency or institutions.^ 

As clearly demonstrated by the text cited above, the four components of 

the Project were created as a means of responding to the problem areas defined. 



*For a clear and brief definition of the goals of each of these four areas of 
the program, the reader is referred to Appendix 0, which is taken, from', the re- 
vised proposal submitted by New' York University to the* National Science Foundation. 

2d 
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Those problems center around the need for improved teacher training, better 
instructional practices, a more informed research effort, an d an improvement 
in the way resources are brought 'to bear on dif f iciflties that have been de- 
fined. A? will be evident "jthrpughout , the evaluators fceel the m£st equitable 
practice in stating Project objectives and clarifying intenr, is to allow .the 
documentation to speak for itself. Summarizing^ the overall purpose of Project 
City Science, the following excerpt from Progress Report #11 seems to offet' 
the most conci$e explanation of both the immediate and long-term purposes of 
the program. ^ ^ 

As stated in the Project City Science revised' proposal for 
refunding, the Project is committed to the establishment of 
four products: two model districts, a unique PreserVice 
Program, a research institute for the study of, inne^-city 
* science, and a well-articulated model for 'change and in- 

stitutionalization. Furthermore, activities Ghdertaken 
' which fall under each of these rubrics would be ones which 
lend them/Elves to visible entities with dissemination 
'capabilities. 'Clearly, from its inception the Project has 
had a wide scope in mind, with the hope of having its model 
for educational reform adopted by other njajor Tftiifersities 
and theii: neighboring school systems throughout the nation. 
Indeed, this notion is contained in the phrase, mission- ^ 
oriented Project. To accomplish this broad goal calls for 
communication with university researchers and. administrators 
and the administrative and teaching personnel of., school v 
sys terns. ±y 

The proposers of the Prpject set Very important goals for the program. 

The Project had high expectations for what it could accomplish in its jLmmedi- 

/' ; t , 

ate environment — the schools of New York City. B.eyoad that, the hope was to 

establish models and assemble data that wou£d_be of interest and-us«--to the 

broader' community of science educators. 

As noted earlier, tjjje PCS Project Director -believed that the prdgram 

could help "effect a nevKkind of science te^hing." * Each of the four major 

components of the ^program were intended to meet no£ v of&ly local, but broad, 
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document filed with the State of New York described 
as follows: r 

The preservlce Intermediate School 'Jeaching Program is \ 
taking form, acquiring character* ag4 before long should 
have established itself , nationally as 'the highest quality } * 

program of its kind. 2^ . ^ , ' «. j 

Similarly high expectations were held for each of the remaining major, compo- 
nents of the programs. > t ** 

Dissemination: ' - f * , . 

'We're disseminating what we learn. Eventually we'll have a 
national network of city sVhool systems that have access to 
what we've developed and we'll have documentation for J them 
to go to.^ - v 

Model Districts: 

We propose to have within three years two school districts 
operating in such a way as to stand as visable, visitable 
examples of what can be attained eveaJLn- the face of inner 
city economic and political problems. 

Research: , ~- 



A comprehensive research program to analyze instructional 
problems and offer broad solutions (is part of the program). 
The intent is to design a lasting mechanism that will begin 
to make headway in generating systematic knowledge about 
the science learning of eatly adolescents in the inner city 
. situation and ai^o about how to achieve science teaching 
in the innfer cit^sehD^ls . ^3 

The task the Project sought to undertake; was a serious and difficult 

one. The goals set were broad in scope and often quite complex in dimension 

Even following twd years of experience and facing a reduced budget, the Project 

leadership appeared to feel the accomplishment of /the major goals originally 

set for PCS remained within reach. 



4? 



Appendix P offers th^ full set of Project goals and a list of attendant 
activities related to these goals, drewn from the proposal submit t^d by 
PCS to implement Phase II. 
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D, Evaluation Plan and procedures 

The assessment of project City Science addressed itself to the major 
priority of the program, the effort to improve science instruction in the urban 
intermediate school environment. The evaluation followed a "responsive" 
approach pioneered by Stake and others at the University of Illinois* In a 
responsive evaluation, considerable emphasis is' placed upon close observation 

of the program being conducted and continuous interaction with Ptoject staff 

* 

and others participating in its functions or served by them. Primary a^tten- 
tion, is given to the activities and communications of the project, identifi- - 
cation of major issues related to these activities, and the collection of 
relevant data upon which judgments can be based. 

To. accomplish these purposes, the evaluators employed a design composed 
4 of four basic elements: A program of regularly planned observations, a series 
of interviews conducted with key participants, the collection of questionnaire 
I and survey data, and thorough review of Project documentation. Once the design 

had been developed, an advisory panel was formed to offer the evaluation team 
an outside perspective. The advisory panel consisted of Drs. J. Myron Atkin, 
Jerome Notkin, and Vincent Reed. They provided /he evaluators with expertise 
in the areas of program assessment, science education,' and school administ^|t- 
tion. Members of the .panel visited PCS field-sites, observed and interacted 
with Project staff at NYU, and consulted with the evaluation team- at regular 
intervals. ^ 

The program of planned observations constituted an- element of majol: ira- 
portarjjee in' the conduct of the evaluation. It included regular visits to 
.classes conducted for pre-service interns at the .University, monitoring PCS 
staff meetings and also attending meetings of fnterns and 'smaller 'grpuVa_>^ 
of Project staff. Evaluation team members were also in attendance at several 
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conferences where Project personnel discussed activities or presented papers. 
Observations were also conducted at all Project field sites on a regular f 
basks. ObservatiQn visits yer$ not^bhfined to members of *the evaluation 
team. A balanced set- of visits by science educators working at the college 
level, science supervisors from the public schoofs, and classroom science 
teachers' was also, arranged. Observation protocols containing evaluative 
comments or written reports were requested of all observers. Observations 
were conducted throughout the-^course of the evaluatio^and dfeserver^ were 
sent to both University and public^chool classrooms. ^Sorae videotaping of 
instruction in the schools was also conducted. 

An equally extensive effort was r raade to continuously interview those 
connected with the project. Interviews were conducted with all Project 
staff r *interns, and scKpol personnel over a two-year period. Both structured 
and informal intervi^wjjrocedures were employed. A number of interviews were 
taped to allow a more accurate and reflective appraisal of views expressed. 
In addition to those immediately involved in the Project, interviews were 
conducted with New York University officials, members of the United Federa- 
tion of Teachers, PCS advisory panel members, NSF officials, and various 

schobl district and Board of Education personnel. 

* t , 

Questionnaire and survey data was collected from pre-service interns, t 
on-site, coordinators, teachers and administrators in tjrfe public school, PCS 
Advisory Board members, metropolitan New York and other; selected colleges, 

j 

recipients of Project publications, former PCS staff members, interns who 
had dropped out of the program, and all recent and former Project graduates. 
" , A case study was also conducted of the PCS involvement in another school 
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district during its initial two years of operation. The bulk of the data 
referred to above is presented *in Appendix Q of the full report. 

The last major element of the Evaluation consisted of ^an ongoing assess- 
ment of all Project communications. This effort focused heavily tfpon Project 
publications, -particularly the Progress Reports, but also* included the original 
and revised proposals for fundings Advisory Board jhinutes, internal corres- 
pondence, course. outlines, papers presented at conferences,, communications to 
staff and school officials, agenda for staff meetings, and a variety of 
external correspondency dealing with Project concerns. 

The data collected from all these sources was presented and^iiscussed at 
evaluation team meetings. The results formed the basis for planning succes- 
sive stages of the evaluation. In terms of the view which are offered, the 
evaluators make no claim to infallibility. We "can only state that we have 
attempted to insure that the important observations we offer are well-supported 

• 0 

We believe they represent an accurate picture of the Project but understand 
that others, including the PCS staff, will find points of disagreement. As 
we have noted, the methodology which was' employed placed a heavy emphasis 
upon the collection of observation, interview, and survey data. Tjie conclu- 
sions we have reached have been extrapolated from such data with* care exercised 
to be sure that th^y were confirmed by more than one source. It is our sincere 
hope that what has been assembled will prove of some use to the Project staff 
and others interested in making similar efforts.. , t \ 



* . ir. OPERATION OF, THE PROJECT • 

A. StafYing* and Organization f 

As we have no tect earlier , the accomplishments of the Project haSte been 
. 'limited by the size 6f the staff that was available. The problem this posed . 
,^^s nftt confined *to staff size alene. -That is, it moved beyond the sheer 
limitations of numbers * to present further complications in regard to the ~ y 
variety of talents and skills that were available. In a project that was 

* *. " r * 

v 

v. working in as complex and demanding an environment as this one, - there^was a 

■"'••V*. 

constant need for a diversity of insight and understanding that vas neces- 

' \ ■■ . • • 

sa'rily limited in- so small a faculty. Given the ambitious intent of the 
• program, it seemed that the scope of the talents needed was always broader 
than* that "Vhich wa^ available. -« ^ 

- To point^to su(/h limitations is not to $aM x a » implied criticism o| 
the professional ability of the staff that was ^empljyed. Admittedly t the' 
ambitious gaal^ of the Project did place a* premtfanr upon the intelligent 
hiring of staff and a careful match-up^- of their skills to the tasks that . 
needed to be performed. I That issue should riot- be avoided and will be ad- 
dressed in a later section. .What is alluded to here is the simple'lack -tff 

human" resources and. the restrictions this posed for, a project with such 

\ 

a diverse set of expectations. The Project had available to it the equivalent 

of three full-time faculty positions. In attempting to meet the many demands 

upon them, these positions were^a^various times spread out over as few as ^ , 
' ' , "^>^ * 

siic^jrid as many as nine part-time faculty members. While that met the need- 

^ ' f . " ' 

to expand the set of skills available, ±ti left each member of the §taff with 

mother sets of tesponsibilities that requfred their attention. * - 

Faculty" members working on the Project were to be relea'sed from an equiya 

lent portion of their normal responsibilities. It is not easy 'to determine 
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precisely the set oj responsibilities. of which Project staff was relieved. 
The number of faculty in the Department of Science Education had heen de- 
creasing fir several years. Because of this, it is hard to determine how 
many persons were needed to conduct the normal business of the department, 
aside from the concerns of the Project. The separation between the Project 
and the department was not clear. In certain ways this was intended and ( 
necessary. To accomplish the intent of institutionalization, the lack of 
separation was good, but in practice it meant that the remaining staff had to^. 
conduct all the ongoing business of the department as well as that of the 

0 

Profect^^The organizational strain was obvious and was noted by PCS staff. It 
frequently appeared that PCS was added on to the normal set of departmental re- 
sponsibilities, with key staff members assuming heavier loads to accommodate the 
needs of the Project _cather than being relieved of other tasks. 

.Because of the lack of sufficient resources, there was a considerable 
premium on skillful o^gamri^tion, administrative leadership, and inter- 
departmental cooperation. In the opinion of a number of the former and some 
of the 'present staff members, these important conditions were not always 
satisfactorily met (See Appendix J). Much of the inter-departmental effort § 
that was planned never came about. A portion of this was due to conditions 
at the University, discussed earlier. This, however, cannot account for the 
almost complete lack of cooperation frc>m other departments. Certainly, a part 
of the lack was also- due to the barriers to cooperation which normally exist 
between departments. A certain resistance to such efforts, particularly when 

**** > 

they are under the- aegis of a single unit, could have been anticipated. A 
reasonable plan or design for dealing with such territoriality was necessary, 
but apparently did not exist. That represented a failure of foresight on the 
part of leadership, thereby denying the staff some much needed-help . 
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The Project was also handicapped by changes in administrative leade^hip. 
The influence of such. changes cannot be fully evaluated, and it would be 
wrong to attribute more to them than is reasonable. Nonetheless, such changes 
at important junctures can and do have important consequences. The actual 
shift in leadership often is preceded by^a period in which the impending cha^e 
is anticipated, and the complete attention of neither the incumbent nor the 
incoming leadership is fully available. That can be disruptive, as can the 
-normal change in leadership style and emphasis, in the case of PCS, the 
staff operated for a year with a Project Director who was present only two 
days a week. Thus, to the complication of a change in leadership was added the 
n4ed for a certain division of administrative responsibility. That too pro- 
vided some difficulty. Beyond this, since the- Project was predicated upon 
an evolving definition of roles, it may not have been prepared organiza- 
tionally to handle the types of problems it encountered 

After the New York City crisis (and perhaps again i^ter the change in 

leadership), there appeared a need to^ reorganize, find new'resources, and use 

the staff in different ways. That this was not done seems, in retrospect, 

unfortunate. Surely the climate in and around the Project at that time ha<J 

•changed drastically. It was no longer clear that the original goals re- 

maihed within reach, but it seems certain that they were not attainable in 

the same way or. f.o the same degree that had initially been planned. That an 

extensive reassessment of purpose was not conducted should not.be overstated 
» 

as a fault. Considering the extent of the crisis, one could reasonably 
conclude that the PCS staff responded well^nough to have survived/Ll remain 
a viable f.orce y after is. was over. Still the choice was possible. 

For the benefit of those attempting future efforts of this type, it may 
be useful to consider what alternatives were available under the circumstances. 
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It would seem that the best counsel that might be offered a project staff facing 

Changes as extensive as those which confronted PCS would be a recommendation to 

consider a thorough reorganization. This would include a complete re-examination 

* 

of purposes and of the structures that had been created to accomplish them. It 
simply do^s not seem appropriate to continue applying a construct which had been 
created during one period to another period for which it may no longer be appro- 
priate. It could be that the superior c^uTs^Qf^ap'tion in such an instance 
would consist of simply marking time while reassessing what is possible. 

.We believe that such a course of action would have benefited Project City 
Science.' It is not that the failure to employ it emptied the Project of pur- 
pose or direction. *The staff continued to do useful things. There is a real 
* possibility, h6wever, that allowing the staff to remain fully engaged through- 
out the crisis was a mistake. PCS offered the schools a welcome service but 
at the possible expense of denying its own staff 'the time they" needed to 
consider how-to redirect their own efforts. A temporary halt might have ^ < 
allowed the PCS staff to react better tp the changed circumstances and to even 
find means of capitalizing upon some' of the unique opportunities/these changes 

» » " * 

may have provided. 

' * We noted earlier our concern about the Project staffs tendency to set 
goals at such an unreachable level that the practical operation of the program 
was conducted without real 'reference to them. The Project appeared to move 
increasingly' into an informal mode where the relationship between behavior 
and purpose was not always clearly charted. The crisis described would appear 
to have reinforced that tendency. A temporary halt would have provided one 
means of rectifying the problem, allowing the staff to plan a better fit be- 



tween what was intended and what now remained possible. 
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B. Modifications 

In an earlier evaluation (see Appendix A), we noted a modification of 
the Project in .which the formal model for institutionalization and change was ' 
apparently deemphasized. and the dissemination effort expanded to replace it, 
The shift in emphasis may have been dictated by a number of changes in the 
climate of the Project at that time, though there is little record of con- 
scious planning. At that same time, however, the Project faced another set 
of circumstances which did seem to demand that important modifications be 
made. Having shaped its design and submitted an accompanying budget for the - 
conduct of Phase II of its effort, the PCS staff found itself confronted with 
a fifty percent reduction in the funding that had been. sought. One would have 
expected an adjustment in some of the broad aims of the Project which would 
have corresponded to the new set of realities dictated by so severe a cut. 
In our opinion, this did not happen. - 

The Project staff sought to make its adjustment mainly in one dimension, 

that of reducing the nunjber of school districts with which they proposed to 

work. The revised proposal senf to NSF noted the need to "reduce staff and • 

\ _ ' - 

to eliminate or cut back certain activities." It goes on to indicate, however, 

that the major modification would, be to substitute "intensity for extension," 

explaining that Project staff would work in two school districts rather than 

the four that had been originally proposed. What they did not attempt was , 

to reassess the broad ambitions of -the. Project and whether or not they could 

still be accomplished". That decision seems crticial. In retrospect, it appeals 

f * 

that 'the Project staff did not give sufficient thought to how the budget, cuts 



A *As will be shown in a lat6* section, the-change model was not completely 
'abandoned. Attempts to formally apply it ware discontinu^ but some elements 
of the initial effort still remained. 

* 9 
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would affect their ability to perform a number of the difficult tasks to 
which they had committed themselves, ^ The reduction was drastic. The analysis 
of jrtiat^was now possible needed tp be conducted at a level proportionate to 
tjiose reductions. It was necessary to determine whether the cuts were such 
that they would influence not only the number of locations but the actual scope 
' of the problems the Project could address. 



\ 



•\ r It surely seems that the influence the cuts would have on the central 
[purposes ofi the Project was underestimated. This may have haa results that 
continued to seriously hamper the effectiveness with which a number of Project 
activities could be conducted. One example of this, previously referred to, 
was the impact of reductions in the total number of staff anticipated in the 
initial planning. Such reductions had an expanding influence on the whole 
operation of the Project, That influence had to intensify when initial 
objectives were not satisfactorily adjusted. 

As one. begins to circumscribe the number of staff who will be available, 
it becomes clear that what is lost amounts to something more than mere faculty 
positions, 'There is a decrease in the scope of the interaction that is possible, 
the diversity of ideas exchanged, and the capacity of the staff to excite each 
other's interest and inspire fresh activity. In brief, the budget restriction 
- posed more than a simple case of reducing the services that could be offered. 
The inner vision of the Project was itself affected. 

It seems thkt two possible modifications would have been entertained at 
this point. One wouUd have been a reconsideration of earlier objectives. That 



What was at issue was not only whether such tasks could be accomplished but 
the quality with which they would be performed. Many remained within reach, 
^but not at the sajie leyel of performance. 
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would involve a reordering of overall aims and not simply the working environ- 
ments. A second consideration would hdve beerV the launching of a concerted 
effort aimed at making up for the los^ of faculty This would have in.cluded 
contacting other sourcesof help within the University to find constructive 
ways to fill anticipated gaps in the Project created by^reduced funding. Thus, 
one of two alternatives would have been expected: Either a restriction of the 

original scope of the Project, dr efforts to find ^Tternative means of per- 

\ 

forming functions that would otherwise be detrimentally affected. Efforts at 

i 

ad justment ; were made, but not at a le\^f consistent with the budget restric- 
tions that,' had been imposed. It seems that the Project staff 'sitnply miscalcu- 
lated its own needs by assuming they could attempt what had originally been 

intended even when faced with such a grievous reduction in funding. 

t 

C. < Support for Implementation * 



In initiating' a discussion of this type, the evaluators feel compelled to 
emphasize the intense difficulty faced by a project working actively in the 
schools. Much of what takes place in education conspires against such risk- 
taking. Funded programs, operating in that same milieu, are far from exempt. 
It is easy to fail when there* are so many elements over which no real control 
can be exerted. The broader the ^nf luence_ that is sought, the greater the risk. 
It becomes increasingly simple to have important things go wiong. The possi- 
bilities for interpersonal disharmony are greatly multiplied when one considers 

the variable of inter-institutional -cooperation. Conflict can erupt both 
within and between staffs. Inexperienced intern^ or- faculty can use bad 
judgment that reduces or hinders Project acceptance. The prospects fpr 



The PCS staff has referred to the need to achieve. a "critical mass" in reference 
, to its research effort. The term is lacking in specific meaning but the idea it 
attempts to ^convey is applicable in regard to the potoj. of faculty talent that 
projects such as Xhis require. 



failures are numerous. 

At the interpersonal level, this Project has handled ifs relationships 
with the schools quite well. .The districts in which they are currently working 
want them to return. Their efforts are generally^ respected and their relation- 
ships remain positive. This is no small accomplishment, and it has left the 
Project in a position where it can continue to work with and influence in- 

r 

st ruction in the schools. 

In managing its field relationships as well as it has, the Project staff 
has nox had a great deal of external support. As pointed out earlier, some 
portion of this may be their own fault in not overcoming the inertia and 
resistance of other departments at NYU. The University leadership, however, 
has not been of great aid in this endeavor. As at other universities, much is 
offered in the way of verbal support. University administration expresses the 
desire to see its staff involved in the community and offering aid. .The iri^ 
dividual reward structure for faculty and the broad university support systems, 
however, say otherwise. From the level of Dean on downward, there appears to 
have been insufficient appreciation of what was done, or support for what was 
being attempted. While prepared to acknowledge the Project's importance, Uttl 
in the way of either material or moral support was offered. Despite^ the rather 
clear nature of the need for University support implicit in the proposal, the 
Project appeared to be very much on its own. Not even its success in managing 
its field relationships so skillfully seemed appreciated or much valued. In 
many ways, the University leadership seemed unaware of the opportunity that 
PCS represented. That opportunity, in our opinion, went well beyond the 

The University continues to apply a one-dimensional reward system that honors 
publication as the sole activity meriting either tenure or promotion. Right. or 
wrong, such a system mitigates against extensive efforts of this type. 



confines of science eduation; It included a bjroad potential forworking in 
the schools and with other community agencies. Jn any event, less seemed 
offered in the .Way of support than would have .been expected. Outside of what 
was funded, little. was provided. It is difficult to see what the* University 
contributed to the Project's efforts to accomplish its major objectives. In 
that respect, it seems that University leadership was far better served by 
what Project City Science offered them than by what they offered in return. 



/ 



III. CRITICAL ASSESSMENT: THE OVERALL PROJECT 



A. Funding Implications ■ * 

The evaluation teanThas stated at several^ points its belief in the Mse^ 
fulness, of this Project- It would seem appropriate to clarify and expand upon 
that belief, particularly in the, light of some of the, criticisms that have 
been offered. PCS. constitutes an 'important effort. On a broad level, it 

* < ' 

represents an attempt on the part^of' the 'funding agency -to determine- the 
efficacy of concentrating resources, focusing upon the support of large-scale 
efforts rather than a series of smaller ones. The final estimate of .such* an 
outcome will take some years \o determine. It is unclear &t this point whether 
this Project will continue, or if it does, precisely what form it will ta&e. 
Time alone will allow a determination cff th$ Project 1 s-long-range success in 
1 finding its own sources bf "funding and establishing its importance as a voice- 
for science education. & t a t 

One can*, even at this juncture/ JioweVer-, offer some estimate of the 

3 . * 

reasonableness of such a funding approach. The evaluators, from* their present 

\ ° • 

•vantage point (and we admit there are important limitations to whfrt we can now 
see), believe that tt$f experience of this Prpject raisas some important ques- 
tions which need ta be considered when funding major, endeavors such as this. 

V Q z 

On the one hand,- the> value of supporting projects witrh the broad capability of 
a PCS is recognized*. • That support allowed things to be attempted that thirty 
smaller projects £ould not have done. On the other hand, 'the PCS effort has 
' given reason to beljgve that such large-scale attempts often bring with them 
problems that are not easy to overcome. .For example, when one launches -an 
effort of this magnitude, it seems that it almost inevitably results, in £he 
creation of . a complex or sophisticated model. Such models are, by their very 
nature, often a% variance with the systems into which their proposers seek, to 



have them introduced. They are also efforts whose actual functioning often 
depends upon a partnership, but which have usually been exclusi^y^f conceived 
and developed by only one of the partners, the university. As noted in our 
earlier evaluation (see Appendix A), the inherent inequality of such partner- 
ships frequently 'results in inter-institutioi(al working arrangements that 
frustrate attempts at ma-king permanent change. These twin dangers* seem 
general, and future fundfing should be predicated upon a clear /demonstration 

that they have been consider^ and that compensation has been made for their 

■> 

disruptive potential- t - ' • t „ 

As is obvious, an organization such as NSF must make a number of important* 
decisions about how funds should be allocated. Discoveries about the limitations 
of a particular tyge of funding in no way indicate a failure. Such efforts allow 
important understandings to ytTe reached and, thus^ irrespective of results at 
the Operational level, something of value will be learned from .projects such as 

I / 
this. ' 

in "regard to the funding of programs with broad intent, we believe the 
potential limitations we have not§d above are serious and require attentidn. jf 
While we fully support tfhat has -been atteihpted through the PCS venture, be- 
.lieving it an effort that should hav^ been made,*we are' not at all convinced 
that we would recommend' that other such efforts be funded without assurance 
that some of „ the relatedproblems have teen confronted and dealt with. 

Project City Science is important at a number of other levels and these 

A- „ ♦ 

also need to be examined. The Project has provided v an opportunity to discover 

the extent to which large urban school districts can be inf luenced^ and whether 

different teacher training models can produce unique and even transportable 

c ft 
•rjesults. As in the case of funding procedures, what can be learned is not > * 

limited to a narrow definition of "success/ 1 What the evaluation is seeking 

36 . " ■ 



v ; 



-31- 



to determine is not simply how well a particular aspect of the Project has 
succeeded \ but what has been learned and' how effectively it has been reported. 
In «such a view, even "failures" that lead to the raising of significant 
questions are valuable. ^ • 

Because this* "is so, the evaluators have chosen not to shrink from a 
critical examination of issues both large and small in regard to this Project. 
As will be evident, we feel PCS has fallen short of its expectafions at a number 
!of levels. Nonetheless, the attempts," if openly exam^neS by either the evalua- • 
tors or^fce'PCS. staff , constitute* an important parfc of the learning that can 
evolve.*. The danger w£ have tried to avoid is that of excusing the Project's 
failure" to add to our knowledge by pointing to the hardships it faced- or by 
focusing upon the nobility if* the effort. That would be unfair to both the 
effort made and the results intended, .The Proj-ecf accepted 'the serious respon- 
sibility of attempting to further inform th"e~ science education ttfmmurfity on a 
number of important issues. The evaluators have attempted to give tjaat charge 
the serious attention it deserves.. 



B. Instructional Methodology 

9 

How effective is the classroom instructional model the Project is seeking 
to see implemented? We believe the Project has not made a significant contri- 
bution in this regard. Little that would add to the depth of our understanding 
about the effectiveness of the -"hands-on" approach, or that would improve x^ur 
use o|^it, has been reported. It has not^>een the subject of serious research, 
of even extensive reflection by the staff. To a large extent, Project per- 
sonnel seemed to^begin and conclude their efforts with the assumption that 
such an approach was implicitly superior. Far. too little was done to explore 
thtfse 'assumptions in any detail or to offer Supporting evidence. 
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One of 'the real problems with a^hands-on" approach is. that students often 

bring jt 6 it a personal agenda that is non-lesson related. They cW to the 

experience which has been .prepared by the teacher with their own soc\al and 
* * \ 

psychological need5, ' A£ least some of these will be directly cdntrary to the 

iqtent and the requirements of the^ learning that is' being offered. Under such 

circumstances, the very structure of the "handsr-on" approach can invite a con- 

flict of purpose. Subtle responses and adjustments need to be made by teachers. 

The difficulties inherent in the, approach must be candidly assessed, analyzed, 

and^ealt with, not ignored. Too much of "tjamds-on" teaching is built 

upon an assumption that interest and the need to- learn will so far outweigh 

* 

£ther considerations that those other considerations do not really require 
serious attention. , There is far too little evidence that thjis is so, and the 
entire approach needed to be explored rather than given an Jj priori acceptance, 

I.t is generally acknowledged that this teaching method; rests upon the 
intellectual" 1 curiosity or' at least the interest of the student. What is not 

so easily seen is the great difficulty involved in instructing teachers in its 

* r * - 

use. It is hard to understand exactly how one is , "trained" in such an 1 approach. 

It has to be experienced, understood, even lived by tH& teacher before it can 

' \ " ' 

be passed on to anyone else/ personal -acts of discovery are just that; ;Lf the 

teachers have not themselves experienced, sucji acts with considerable?, frequency , 

they are ill-equipped to guide students in making them. This' is'as true for 

the university teacher as it % is for the intern. Too often, trainees attain 

the vocabulary without having experienced the process either deeply or often 

enough. Worse, they are left unaware that this is so. 

That PCS dj4 not- entirely overcome the dilemma posed by this instructional 

r 

approach is attested to by the relative infrequency of its use. The\evaluators 
did not observe a- great deal of n hands-on ff instruction anywhere, including the 
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uuiversity classroonfe. Some eighty to ninety pertent of what was observed at 
NYU was teacher lecture or group discussion. It was not common to see the 
teach^tfg model so often spoken of actually practiced. The field experience was* 
roughly similar with regard to the frequency of "hands-on" instruction observed. 
Observers in the schools, to a large extent, saw a practical reinforcement of 
the instructional model most frequently practiced at the university. The PCS 
staff itself lacked a master teacher who could demonstrate the model in suf- 
ficient variety and detail. Most of the instruction viewed was* quite tradi- 
tional in nature — an observation shared by thejmajority of visitors to either 
Project field sites or NYU classes. (See Appendices F-H.) 

Perhaps if the approach had been more intensely examined and 'its possi- 
bilities more closely explored,' greater use of it might have been seen. If 
the PCS staff had from the beginning treated "hands-on" as an hypothesis to. * 
be testei, more of importance could have been learned' about the specific con- ^ 
ditions under which dt does or does nat work. By assuming that it. was the 
best instructional intervention available, PCS has left the science educati 
community with the same questions, the sape lack of knowledge, and the same 



gaps in its understanding. Too little help been learned. from five years of 
Project experience in this regard. J 

- ^ j 

C. University-School Interrelationship^ . 

* .' ' » - • 

The schools" serve* a number of important purposes, and balancing them 
always presents a problem. The socialization function of the school" vies with 
the .educational, and* .of ten a real conflict is posed. Teachers, sometimes 



*A number of science teachers in the schools in which PCS Was located consistently 
reflected the view that as an instructional method, "hands-on 11 * was more suitable 
for able students. Views such as that needed to be systematically' examined, 
probably by the research arm of the Project. Uhfortunately, they were -not. 
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unconsciously, struggle to maintain what is in their view a reasonable balance. 
Students are increasingly- peer group oriented, , and place great premiums' upon 
the need to be liked and accepted. Thus, the sofcial aspects of school life 
compete with the academic for the student f s time and energy. Some of the 

' methods posed by teacher-training institutions inplicitly require a considerable 
increase in the degree of socializing allowed* Teachers resist this, and perhaps 
they should. There are some deep and„yital questions here that need identifi- 
cation and discussion. The univegr$ity too often moves directly to answerg 
rathen^than^explorations, neatly ignoring some ot the jreal problems. The 
schools, whicnS^ave to deal with consequences, can afford no such luxury. The 
larger educational community needs 'to 4 face the fact that schools ignore the 
advice of university teachers and researchers not because they are fgnorant, 
recalcitrant, or both, but because they do not believe the real issues *are • „ 
being addressed. That belief may not be as inaccurate as critics of the schools 

• 4 wou£d Rref^r to think, 

* ** 

.Part of the difficulty is that few formal mechanisms have been established 
that would facilitate <a real exchange of ideas between the school and the 
university. PCS attempted to create such a link in the form of an on-site 
coordinator^ This did not prove to be a completely satisfactory mechanism. The 
coordinators 1 prof essional , standing did not seem strong enough for members of 
either institution to utilize them for such a purpose. That would have required 
more teaching and/or supervisory experience than most persons who filled that 
position possessed. Their status appeared to leave them ill-equipped to 
address institutional differences with authority^ as several of the coordinators 
themselves pointed out. A corresponding difficulty was posed by the fact that 



Tt app^eared obvious to -several of the coordinators that in their capacity as 
key implementers of change in the schools, they were often not taken seriously 
by ,e±ther group. 
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three of the eight coordinators this year were employe.es of the public schools. 
Due to the nature of their responsibilities, the degree of interaction with 
PCS appeared extremely limited and little was done to use any of them effec- 
tively as a liaison between school and university, 
j. In the absence of an effective mechanism, resistance to implementation 

efforts is likely to persist. The schools continue to offer opposition and 
those in the university- continue to offer convenient rationalizations for the 

i- 

, refusal of others to adopt their ideas. To accept their explanations would 
require tHe belief that .the reasons for rejection are almost always unsound. 
^ # That seems unlikely, and so deeper causes need to be sought. While teachers 

do not always take the time to offer a formal analysis, the constant refusal 
to adopt a particular approach should be viewed as something more than simple 

' • obstructionism on their part. There may be a basic flaw in what is being - 

i i . 

suggested: a real conflict between it and the environment into which its 

* * 

incorporation is being sought. The responsibility for discovering the con*- 
flict lies with the university; not the school. Since the university advocates 
the suggested 'change, the corresponding obligation of demonstrating the proof 
of that change's virtue remains theirs. It is an obligation which has not 
frequently been accepted — indeed one that often appears to have been studiously 
avoided. It is a simpler task to condemn the schools, attributing the lack of 

acceptance to their extreme- conservatism rather than the university's refusal 

— * 1 * 

to give up sifch comfortable rationalizations and seek deeper, structural 
courses. * 

There are .two conservatisms at work here, but only one is generally identi- 
fied. The university has for -too long been expert at identifying the resistance 
to change in other institutions while remaining blind to its own entrenched 
patterns. It remains too" tolerant of behavior that is comfortably -self-serving, 

41 . ( 
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but has proven neither productive no^-f ruitful. PCS has been prey to this 
problem. The Project began with a heavy field emphasis. In the beginning, in- 
service workshops were conducted in the schools, and university faculty made 
frequent visits to field sites. Over the duration of the five-year funding 
period, however, one can chart an increasing withdrawal of the faculty to the 
confines of the university campus. This seems part of a larger pattern for 

e 

such programs. The staff began with an emphasis upon working in the field but 
increasingly yielded to the temptation to^ return to an environment in which 
they felt most comfortable. There is a pattern of gradual withdrawl from what 

r 

was the central arena of intended activity. During the last year of this 
Project, there were increasing complaints about the non-availability even the 
on-site coordinators in th$ schools. ' Thus one is witness to the retreat first 
of front line university faculty, followed by what would consitute second line 
staff. The schools note such withdrawal with more than passing interest. 
That they draw appropriate conclusions about whether the university's ideas 
are workable in their environment cannot be doubted. 

The universities, thus, have their owyl problems to overcome. They often 
employ a one-dimensional approach for dealing with the schools that is in- 
consistent .with their expectations of a flexible response. While the problem 
referred to above requires resolution, the overall dilemma is far from unique 
to this Project^ That it was not overcome is not surprising. It is not noted 
as a special flaw but as part of a constant and continuing limit to innQvative 
efforts initiated at the university level. Those employed there need to take 
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The lack of power, authority, and status within the schools are probably impor- 
tant factors in thfe gradual reduction of the University presence. The capacity 
to influence always seems greater when viewed from dfar. 

The coordinators were not considered faculty. 

' c 
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a broader and more embracing view of the university's own conservatism, its 
resistance mechanisms, and its "hostility to modifying long ingrained practices. 
They' need, in short, a far more sophisticated~approach to attempts at coopera 7 ; 
tive interaction with other institutions than they currently .possess. • \ 

The interfacing of institutions is a complex and demanding process re- 
quiring an understanding of institutional behaviors, and the development of 
distinct .-strategies or approaches. The NSF would be well advised to require a 
demonstration of such an understanding (and a well" designed plan^to put it into 
effect) from any outside institution seeking funds to implement programs in the 

<5 

schools. This Project made reference to an institutional change model in its v 
proposal, but never appeared to take it seriously enough ( to formally plan and 
apply* it. Even then, the model was too "other-institution" oriented, reflec- 
ting an ethnocentrism that was ill-equipped to deal with the full set of 
problems inherent in such inter- institutional arrangements. For the scope of 
what was being considered, PCS took too little account of the type of resistances 
they were bound to encounter. Even less thought was given to those to which 
they t-hemselves might prove susceptible. The result was that the implementation' 
effort was at best uneven, frequently uninformed. 



D. Effecting Change 0 

The Project had, as part of its design,, the calculated use of key individuals 
who were to serve, as agents of change in the schools. PCS had assigned this role 
in different ways to both its on-site coordinators (OSC's) and its interns. The 
.strategy seemed to be to work at changing conditions in the schools by skillfully 
employing the talents of individuals filling those two roles. The OSC's, being 
professionally more experienced, wer a generally charged with greater responsi- 
bility in this regard. ^ 
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Based on observation and interview' data, a reconsideration of who would 
serve best as the agents of change is suggested. The evaluators would urge 
that the PCS' staff contemplate* training-school personnel to fill this role. 
In particular, we would suggest working with administrators at the building 
level* and science coordinators (or other key personnel - ) 'from the central office. 
There is a need to interact with these individuals and discuss questions of 
deeper educational significance. By th§ir own admission, school* administrators 
have all too little opportunity to do this. They need to be invited in as joint 
partners: co-planners of key Project efforts. They need to be consulted about 
direction and used in a way which would help them see themselves as conscious 
agents of change, with a particular purpose and an accompanying plan in which 
they have confidence. This will require developing a strategy with them, and 
clearly defining their role. Including. them in such planning would^ not only- 
employ their talent and experience, but would provide the opportunity to inform 
and direct- the key implementers of such^a strategy. This would appear a more 
promising approach than that which has been\attempted. 

The training task is not an easy one and needs to be carefully considered.^ 
The resulting application, would be far more direct, however, Further, if it 
proves workable, the potential dividends are much greater. One can make a 
real beginning at the development of model schools through such an approach. 
These administrators have a large say in employing staff and often set the 
standards' for the school. Their normal institutional role allows them to hire 
teachers, thus using their position to^improve the quality of the staff and 
change the tenor of the school. 



*This includes interns graduating from the PCS program who could presumably 
provide the foundation upon which revitalization would be built. As noted in 
an, earlier evaluation (Appendix B) % the Project failed to give much attention 
to seeing that their graduates were so employed. 
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i PCS appeared to believe that they could minister (iirectly to what they 

felt was a small, well defined population: the junior high school science 

teachers • In retrospect, it does not appear that this was ever passible— "at 

least not in the way that was assumed. These teachers. are part of the total 

# x 

system. They work, within it not outside of it, 'aijd cannot be separately influ- 
enced in the manner attempted. The whole system impinges on^their day to" day 

activity, exacting from teachers a behavior which conforms to the structural 

i 

demands of the. school. In some ways the Projec^ staff seemed to }cnow this, 



but in important ways they did not act upofi it. \t wasn fault in the design 
that was never fully compensated for. The' staff needed a betWr /vehicle for 
making a fuller impact upon the whole system. PCS could have considered how 
to affect administrators and other teachers in order to influence science 
instruction. They chose the reverse route: attesting to influence the system 
through its science teachers. In such competition to influence teacher be- 
havior, 'the, university was badly overmatched. It woutd appear that attempting 
to use the natural structure of the system would have been a wiser course. It 
is possible that middle management, i.e., building administrators and subject 
area coordinators, are the, most influential components in ^ large city bureau- 
cracy. Their tenure is often more permanent than that of either the teachers 
below them or those at the superintendent level and higher. They are clearly 
more a'ccesible, and more, time can be spent with them. They have the capacity 
to mqre directly influence actual classroom instructional practices. That they 

were not worked with more closely seems to have, been a major error. They appear 

i 

to have far more influence as potential change agents with both subordinates 



* 



'Teachers, particularly in large, urban, unionized school systems, tend to 
leave immediately following the last period and are rarely available during 
the day. Superintendents are generally far too busy to give the problem of - 
classroom instruction the attention it requires. 
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and super-ordinates than those the Project chose to use irv this capacity.. 

r 

Whether the PCS staff decides to use the administrators in the manner 
suggested or not, they must increase their interaction with them. The full 
burden of responsibility to effect change cannot be left to interns and co- 
ordinators. These latter are potential instruments which the Project staff 
and school administration might employ only to help implement decisions which 
have been jointly reached. They can serve an important purpose for the Project 
if used this way* If the original role planned for coordinators jiad any chancre 
to work, it was dependent upon PCS employing exceptional individuals in that 
capacity:- master teachers whose demonstrated example was so strong it could 
compel others, to reexamine their approaches. The Project *did not fill these 
positions with such individuals That failure vitiated whatever | likelihood 
of success th^t might have ex'ist^d^ To a large extent, the' Project operated 
on the one hand at too "high a level far meaningful change and on the othfer at 
too mundane a level to be effective. Somehow middle management personnel were 
too little consulted or influenced, and on a day-to-day basis^ they are the 
ones who run the school system. 



*While the staff was able, they could not be described as. master teachers, 
at least not within the* realm in which the Project was working. 
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IV.* GENERAL CONCLUSIONS 
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A- Vision 

^ PCS represents an important and necessary attempt on the part of NSF to 
support innovative educational efforts- Aside from the strengths or weak- 
nesses .of this particular Project, the effort that has been made is important. 
Educators have recognized that the junior high school has been too long ignored.- 
Many agree that it often represent? a pivotal point in the life of students: 
a time' in which vital decisions about their educational futures are being c 
made. For all that educators acknowledge the importance of junior high schools, 

little of practical significance has been done to address or attempt to 

/ • 

alleviate the problems so freely alluded to. 

Project City, Science rfepresents an atjtempt to deal directly with one of - 

the problems. The original concept was sound and the Project's authors did 
$ * > ( 

not avoid the real issues. They chose to work in the schools, dealing J 

directly with students and teachers, thus exposing the Project to the risk of 
failure, in exchange for placing themselves in a position where they could 
make an impact/ The risk was compounded since PCS .chose to work not only at 
the junior high school level but within an inner 'city urban setting that was 
experiencing declining economic , conditions, increasing population shifts, and 
great problems with teacher morale. f 

The initial design for dealing with these problems was audacious in its 
vision. There were some drawbacks to that as will be noted in the next sec- 
tion; a conception so elevated eventually needs to be translated into concrete 

u 

action, and that can prove a stumbling block. Nonetheless, in the initial ^ » 

* 

stages, a grand vision of^what can be accomplished needed' to be offered. Such 
a vision was provided. The design offered was prescient, anticipating inter- 
ests, and pointing to an important policy direction. What was proposed was 
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bold and imaginative. It pinpointed an area of/need, and identified the 
separate resources which could be formed v into an alliance to meet that need. 
The Implementation of such a design was a separate matter., requiring a different 
set of skills. Translating theory into practice offers few guarantees. The ( 
experience of Project City Science has demonstrated again the difficulty of 
attaining objectives that are ambitious in scope and require major institu- 
tional adjustments to accommodate them.- The Project was far less successful 
in meeting such objectives, and some of that- can be tied to the very breadth 

of what was envisioned in the original plan. 
t 

B. Scope 

The evaluators believe that an excessively optimistic set of .expectations 
was, entertained by the Project staff. This was also alluded to by the evalu- 
ators of. Phase I of their effort.* A program that is overly ambitious in its 
intent-ambitious* to the point of being unrealistic can end up*in a state of 

organizational confusion. It can result in subverting project planning £y 

* . • 

'creating a level of illusion that" confuses program functioning, leaving the 

~ staff without clear direction. In such situations a dichotomy fs often intro- 
duced between statements of intended purpose (which the stiff begins to 
conceive of as ideals rather thsn guides) and actual behavior. The result is 
that statements of purpose and actual behavior become increasingly unrelated. 

Such a separation between thought and action appears to have been an 
unintended outcome of this Project. /biased on observer and interview data, the 
connection between expressed Project- purposes and actual outcomes was consis- 
tently' weak. The ambitious scope of the pripos^T research effort, the design 



*The evaluation was conducted by the Center for Instruction, Research arid 
Curriculum Evaluation (CIRCE). See PCS Progress Report 4. 
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for institutionalizing change, and the attempt to /create model districts were 
examples of goals set so high that actual performance was left without realistic 
guidance. This resulted in key Project activities often seeming random rather 
to than purposeful, and discontinuous rather than aggregate in nature. In our 
view the "Progress Reports," while useful, reflect that discontinuity and 
lack of long-range focus. 

Despite the ambitious nature of its goals, the Project remained somewhat 
insular in its actual operation. Far too little outside help was^sought. The 
Project's failure to make use of the New York University community is noted in. 
Appendix B. With the exception^ a portion of a single sociology course 
(dropped in the final year) , only PCS staff were involved, in the actual in- 
struction of interns. -We believe a number of outside consultant^ or guests 
could and should have been used. As one of the interns noted, "It would have 
been good if they had brought 'in a Black or 'Hispanic psychologist who could 
have given us a little better insight about the kids and their problems." 
There were a number of such special circumstances^ where outside expertise 

could have proven invaluable, and not air of it need have been financially 

$ 

prohibitive. Resources from the Board of Education ancf a variety of New York 
City institutions, including other universities, could have .been obtained free 
or at nominal cost. There were, and are, a number of rainority^coalitions, 
alternative Schools and public" service organizations willing to offer services* 

r t 

or provide useful experiences. ■ - - 

'A similar problem was the Project's lack of a broader vision of its pur- 
pose. ' It retained a narrow vifew, often seeing its role in specific New York , 
City terms, and never really addressing the larger issues of inner-city instruc- 
tion in. science. Attempts to put staff or students in contact with science 
educators wprking in similar circumstances were far too infrequent. Drawing 
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upon the experience of programs in neighboring cities was selHom attempted', yet 

the* evaluation staff found *there wer| individuals who were both knowledgeable 

• ** * 

and interested in sharing concerns. . 

Failure to seek these Interconnections may* have been ^n oversight, but it 

deprived the Project and its interns of views thSj^ could h^ve been at once hroad- 

* V 
ening and informative. It also served to deny the Project staff access to the 

potential dissemination outlets they were seeking .~~~Ihere seemed to be too 

limited a sense of responsibility to the wider audience of science educators,. 

Operational aspects of the Project were' conducted < as if tftey represented mainly 

a local effort, with little being tested, developed, or offered in a form that 

would stand rigorous review. A better sense of their relationship to the 

broader -science community might 'have encouraged a more effective use of avail- 
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able resources, and a wider cscope for Project operation. 

\ & 

C. Status - ' * * > 

If one is to be guided by recent ."ijyogfess Reports," the Project appears 

<!> ' ,% ' * : 
to consider that important elements which ar& transfera^^^iave been com- 

pleted. 25 Th£ ^valuators do hot concur. The project is, from our perspective,. 

,mainly h source of potential that has noV ye^t been JEully realized. The extent 

to which it wiLl be remains unclear. Viewed in p a favorable light, xme may ^ , 

consider that the Project is right on schedule. At the end of five years, 

they remain a viable force that is in a position to accomplish, something. 

'^They are, in this view, an unfulfilled but possible sdurce of ^ood in the area 

of science education. Perhaps more could not have been expected than that the 

Project staff would h^ve brought themselves into^such a position by this time, 

although their own hopes were surely greater than that. *, 

Regarding the Project's present status, the evaluators want to amphasize * 
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their view that PCS is not what it appears to be* on paper. .An extensive attempt 
to document .that discrepancy is offered in an^earlier report (Appendix A) . The 
Project's reports, continually- assessing its own endeavors, are,' in the opinion 
of the evaluators, inflated in their optimism about both what has been accom- 
plished and what remains within reach. V 

Projects such as this develop 5 a certain ^isolation from the immediate world 
in which they live* There is a tendency to receive only data that reinforces 
N a positive view of what is being accomplished. Some" of this is natural. The 
work is hard, and positive reinforcement/ are few. It is not a strength, 
however, and needs to be resisted. It was not unusual to find that the Project 
staff discounted viewpoints from sources they considered either biased or 

unimportant. The, concomitant phenomenon was that they were not profiting from 

*> ■ 

what those sources had to say.* The Project was not well served by such 
attitudes. 

While the staff persdnally accepted criticism of their efforts with 
reasonable grace, there was not always a calculated effort on the part of the 
Project as an organization to obtain an unbiased assessment of how others 
Viewed the quality of their work. A weekly hour with interns^ presumably , 
created to obtain feedback, was not well' used in this regard. The Project 
*staff dominated the time with administrative and p^gfam details. The hour 
'was eventually supplanted by a course in the second semester. In the meantime, 
interns complained that their concerns were not' heard. Considering the extent 
of the need for input, this seemingly -casual dismissal of a valuable source of data 



*The attitude was manifested in the Project staff's reaction to the view of 
Some of the school personnel, as well as to their own interns. It is perhaps 
exemplified in the response to the CIRCE report, an -assessment of Project s 
progress which the evaluators believe was informed and accurate in its 
identification of prospective problems. 
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was a mistake and should be rectified. Similarly, the Project's research 
staff was not frequently providing needed information about its operational 
aspects/ In many ways, the Project did not have an external, unbiased ^ 
insight into how it was functioning. 

In the , absence of either openness or access to such external data, the 
Project has clung to a grander vision of its efforts than has been warranted. 
This, in combination with an insufficiently restrained rhetoric, characteristic 
of the Project from its inception, served to blur meaning, obscure purpose, and 
cloud actual activities. It too often, hid what the Project was truly accom- 
plishing, perhaps because it was less than that which was intended or desired. 
The earliest evaluation report said a J much, though sometimes in unappealing 
language. Its counsel appeard to be ignored, and 'the Project staff continued 
to believe they couia^accomplish the grander goals— the larger- vision of their 
purpose. Those earlier evaluators concluded, and the present ones agree, that 
this was never possible, althpugh a number' of smaller but still important 
things were. 

D. 'Planning 

In actual operation; the, Project was guided by the original proposal 

% 

design. We have noted thkt the design was imaginative, and addressed itself 
to real'and important issue^. It was not without defects, however, and some 
of these caused the Project difficulty. A concern has been expressed about 
the scop^.and the depth of the impact anticipated in the original design. 



In this, the /'Progress Reports" were an unfortunate accomplice. The-staff 
was required to issue them at three (later four.) month . intervals . It is 
difficult in sucThja short space^to find exciting activities to report, and 
one is encouraged to use inflated language or begin dwelling upon future 
promises. The .Project staff fell prey to both errors. 
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There we're other difficulties. The proposal called for the deliberate use of 
a number of staff who had no expedience with the Hew York City schools. There- 
was also planned turnover of certain staff members at predetermined intervals. 
The-rationaleTffered was not; entirely convincing* and, }n the view of the 0 
evaluators, the concept did not work well. The lack.of familiarity with the- 
mechanics of the New'York City schools represented an obstacle to a number oS^ i 
the c coordinators' that was nolTwell compensated for by the broader view of 

s > 

events which their inexperience presumably allowed. Indeed, some became 
effective only as they. grew more familiar with the way the schools worked. 
This build-up of experience was often lost to PCS wh^n, either .through personal 
choice or by predetermined plan^ such staff left^the Project. Thus, the weak-^ 
nesses of these positions showedN^far more than did their assumed strengths. 

Another operational difficulty bie Project "faced was the problem of plan- 
ning changes in its own procedures to meet the new challenge's that arose. Even 
a project that begins with the intention of remaining flexible has a .tendency 
to fall into an operational mqld. What' this Project has dejnonstrated to the . 
evaluators is the intense need to hav^ organization planning and objectives 
clearly developed prior to 'program operation. Once the program begins,, a m*jor 
impediW is finding /the time to identify, discuss, and plan necessary changes 
Events begin to acquire a momentum of their oWn, and staff are often in the 
position of reacting to^ father than drr/ctihg them. 

"if ptfper preparation is to be made, it is likely to be accomplished in . 
one of two ways. The most likely of these is through the careful and -thorough 



*The use of staff without New York experience was an attempt to avoid a con 
formity to the views of the city system that does indeed exist. The planned 
turnover represented an attempt to view the coordinator s position as a . 
vehicle for training future college instructors. 
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organization of the program in advance.' This includes making provision for^the 
staff to meet at prescribed* intervals to assess, in detail, program progress. 
A second method i*s that of halting .program activity while planning a new direc- 
tion— a' course of action that seems justifiable only in extreme emergencies 

such as PCS faced. If finding the necessary tim§ to plan changes in direction 

< . -\ 

is as difficult as has been perceived^ then a great premium is to be placed 
upon developing a thorough operation design at the outset. 

Important elements of Project 'City Science were not fully in place at the 
initiation of program activity. Some of this was planned. This was particu- 

9 

larly true of faculty roles, which were not carefully described but were 
expected to evolve w^ttr~experience. It was also true of the planning for model 
districts, which lacked detail in regard to key roles or how major objectives' 
would' be attained.. _ The research phase was similarly open-ended, not clearly 
identifying roles or assigning responsibilities. When the operationa l nee d 
for job descriptions that would avoid overlapping efforts arose, there were 
too many demands upon the staff to s allow systematic planning. Weekly faculty/ 
meetings had. to deal with many pressing issues and generated their" own dynamics 
In any event,' $hey would have provided an inadequate forum for the. extensive 
analysis that needed to be conducted. • ' ' . 

In the absence .of ; a formal structure \o which one could resort fpr defini- 
tion of roles and functions, PCS evolved an increasingly infernal operational 
style. This had its strengthen' that it allowed the staff 7 freedom and flexi- 
bility.- It also 'had notable weaknesses: The staff was tieither thorough nor 
systematic in tVeir. efforts. .A clear ^ine of research/was not identified" 
until late in the Project's existence, though a humber of interesting specula- 
tions had been offered" earlier. The implementation of changes in the schools 

was not pursued in an orderly and precise manner." There were not consistent 

/ ■ 
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attempts at raising questions or closely defining concerns. The Project 
lacked an operational persistence, a clear definition of its major tasks, and 
an unyielding determination, to pursue them in a manner that would identify 
important issues, raise substantive questions, or produce information of con- 
sequence to the field. Even the production of monographs portraying a host 
of school-related activities, or journal articles pointing research' directions 
or identifying concerns, would have represented significant contributions. 
Such outcomes, unfortunately, did not result. 

Ort a different level, several consultants recommended that the Project 
employ a number of master teachers who could translafe what was' being suggested 
into classroom instructional practices. This appears to have been a sound^sug- 
gestion. The lack of availability, of such personnel hampered the Project, and 
was noted by a number of* interns and some cooperating teachers. Such practical 
demonstrations of successful techniques would have constituted a potent means 
of changing teacher attitudes aQd initiating the development of model schools. 
As one edacational critic has suggested- 

There can be no significant innovation in education. that 
does not have at its center the attitude of teach.e'rs and 
v it* is an illusion to think otherwise. 26 # ' „ 

• * 

The existence of the master teacher would have been a great aid in allowing the 
Project to translate its purposes ifitoyconcrete, observable actions. 
^ PSC also needed to address the organizational issue of developing effective 
' intra-staff working relationships* ihere>was an operational gap between clini- 
cal and Research oriented elements of the staff. Such differences lare to be 
expected, and can even be productive, but. they must be dealt with skillfully. 
The/ appear to have gone unattended, perhaps unredognized, for too long in this 
Project. Keither group clearly saw the otftefr as a powerful source of aid in 
accomplishing some 'of . their key purposes. 'Because of the nature of Project' 

55. 
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activities and the position of the clinicians, that group attained an ascendency 
in practical decision-making, which weakened whatever research effort may have 
been possible.* Here, then, was. a program attempting to bridge the gap between 
universities and schools—which is wide, traditional, and multi-faceted— which 
had not completely resolved subtle divisions within its own faculty. The 
' result was that the staff was left to implement its ideas without a viable . 
assessment of their impact or value. 

The rift experienced by the sta^ff was a miniature version of the prac- 
titioner-researcher split that has long plagued a number of professions. 
Anything the Project was attempting to accomplish would have p£ overcome not 
only such divisions within its own staff, but the resistance of teachers in 
the field as well. Because such opposition is so formidable, few innovative 
ideas, including this one, find their way into actual classrooms intact. 

Assuming that an invention is born, it -must then find its 
a way into the social network of the practitioner. The isolation 
of the research community from practitioners in education is 
legend. Geographic distances, status differentials (as between 
researchers and practitioners), legal boundaries, and a dozen 
-other barriers inhibit its journey. Most innovations never make 
it so far. Those that do, like individually prescribed instruc- 
tion, are transformed in tfi»rocess. The final metamorphosis 
is performed by the practitioner, who blends the invention with 
other messages and shapes them to his own ends, which are certain 
to be removed from the vision pursued by the inventor. 2 ' 

PSC was not particularly effective as an organization in dealing with this 

common but persistent problem in education. Too much of what was attempted 

yielded to, rather than overcame, this dilemma. v 

E. Legacy 

On an individual basis, the'projefct has attained a measure of success.' It 



*A lack of direction on the part of the research staff was lively an important ^ 
contributing factor in the weak- performance. . , ' 
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has survived under extremely difficult circumstances, and may eventually obtain 

some local funding support. That would be a good first step toward the ten 

n > 

years of non-federally supported activity originally contemplated. Viewed from 
a wider perspective, PCS has* not thus far provided a. great deal that can be 
passed on. The legacy of the Project in terms of particular outcomes is not 
strong. Its disappearance would represent more a loss of potential than any- 
thing the science education community or even the New York City schools would 
immediately miss. They remain an unprove quantity — an as yet unrealized 
potential. ^Their significant contribution, if any, remains in the future — 
not in the past or present. ^ 

Some things have been accomplished. A number of interns have been trained - 
and are working in urban junior high schools.** The Project's publications are 
appreciated and seem to have been well received by some of those to whom they 
have been sent. Most important, the PCS ^taff has developed a working 
relationship with a number of schools, and put themselves in a position to 
offer effective help. Personnel N in the schools report that a relationship with 
the University is' important to them. It challenges complacency, exposes the 
staff to new ideas, and invites reexamination of current practices. As such, 
it represents an association that* has inherent value to the schools. PSC has 
filled this need. 

One must balance these accomplishments with efforts that remain incomplete. 
The model districts are a long way from being formed and the dissemination 



The* most recent information is that the PCS staff has been unable to obtain 
outside funding for the initial years of its proposed Phase III effort. 

Project records in regard to employment of interns are # imprecise. It would 
appear that some 25-30 of the program's 1975-1978 graduates are currently 
employed as teachers with as few as 10 working in urban junior high schools. 
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effort has not yet resulted in the Project f s being duplicated Elsewhere^ 

While the research staff has made presentations at several conferences, 

there has -not been an article published in the five years of Project 

activity, nor is there any prospect that the Research Institute will be 

formed. While the Pre-service Program remains the most fully developed 

of the Project's efforts, it also poses a number of unanswered questions, 

particularly in regard to its power to attract a sufficient number of 

students. - 7 

♦ 

What the Project has done well is to establish communication with a 
number of disparate forces in New York that have an interest in education. 
The staff has developed good relationships with members of the teachers 1 
union,^educational .administration, and several school-related institutions. 
Through the effective use of an Advisory Board, they also have access to a 
varied and highly skilled pool of professional talent. This is no small 
accomplishment. In achieving these things, the Project staff jnay have 
placed themselves in a position where they can be an effective spokesperson 
for science education in ttew York City. If Project personnel can scale 
down their ambitions and organize themselves to do better what they are 
realistically capable of accomplishing, they could make a real contribution. 
Theirs is a voice that can be helpful. What they have attempted is important 
and can be a cons true tiv^aidTVTo the extent that Project staff can focus • 
their efforts on clear Objectives and capitalize upon "the influence they 
have gained, they surely have much to offer. 

In saying this we in no way mean to imply that the Project staff has 
been successful in attaining their original aims, for we do not believe 
.they have been. Nor do we claim they have attained their global objective, 
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' for clearly they have not. They are a spall project doing some interesting 
things, a number of which are commonly done by other departments of science , 
•education. There is one notable exception.. P.CS has made political contacts ' - 
an^ alliances within the New York City educational bureaucracy that are 

' impressive, and unlike those which most comparable departments manage to 
achieve. They have laid the groundwork to beoome a useful and constructive 
force for science education in New Yo&. In pointing this out, we do not 
suggest the Project is what its writtejUocuments imply. It is not. We do ^ 
note that when the overstated claims that often obscure what the Project "is t 
doing have been scaled down, they have the potential to do some interesting 
and useful things. A foundation has been laid and some good may result.. The - 
direction is .not yet clear and the -plan not fully 'formed, but if the staff 

' uses what has been established in an imaginative and intelligent fashion, 
some of what was intended may yet result. Their considerable skill in estab- 

A lishing and effectively maintaining a network of communication^ with important 
elements of the educational bureaucracy can be used to help science instruction 
in the City. That much has been accomplished by the funding. It is up 
to the funding agency to determine whether or irot-that is enough. 

F. Report Format 

As has been indicated, the Project has four major operational aspects: 
the Preservice, Model Districts, Dissemination, and Research programs* In the 
sections that follow, some of the major recommendations that were offered in 
the final report for each of the four Proj^et components' are given. Each 
recommendation is followed by one or two paragraphs of explanatory comment. 



*For a fuller review of the documentation explaining Project purpose, see 
Appendix A of the full report. 
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Following the program recommendations are a list cjf policy recommendations 
which are offered to the NSF for their 'consideration and appraisal. Both 
the full evaluation and the recommendations which resulted ,from it were 
based upon data more fully reported in a separate set of appendices which 
include two earlier evaluations of the Project. Readers .with deeper 
interest and greater endurance are referred to the full report. 
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V. RECOMMENDATIONS 

In this section two types of „ recommendations are offered. The first 
^is concerned with particular aspects of Prpject City Science and is intended 
mainly for Project staff. These recommendations are separated and listed 
under the four program components to which they are related. (Items A-D) 
The second set of recommendations is broader, involving policy matters, and 
is offered to the National Science Foundation for their consideration. 
All recommendations are followed by brief paragraphs in which an explanation 
of their intent and meaning is offered. For a fuller discussion of these 
recommendations,.' the reader is referred to the full report. 
A. Recommendations ,f or, the Pre-?SerVice Program , . . * 

1. The Project needs to exandine the specific skills of its own 
staff and optimize their use. 

A key' to the success of any Ptoject of this type lies in the proper use 

of staff skills. It itf clear, for example, that not all staff members functioned 

equally well in a teaching situation though they may have had compensating 

strengths which could be well used in other areas. The Project's staffing was , 

weak at several key points in that staff skills did not match up well with 

the set of ^responsibilities and tasks that needed to be performed. 

2. The Project must develop better means pf obtaining information 
from their preseryice interns. 

In the p^st two years, a variety of highly useful information^as been 

collected by the evaluators from on-site coordinators and preservice interns. 

Hi^ same type of information could have been gathered by the Project staff" and 

used to shape their professional efforts. The project had the services of the ( 

research staff and a better effort could have been made to not simply test the 

interns but to also seek their inputs in a formal and cons is tent, fashion over 

the course of a school year. A similar problem existed in regard' to following 

up program graduates. ?ar too little was attempted and an invaluable source of 
information about the project, was lost. 

61 ' ' 
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3. 3he university must learn to separate Project business from the 
business of the department.. 

The Project is a full-time venture. The University 1 s continued demands 

upon the department helped deplete the Project staff's time and absorb their 

energy.^ Because of financial arrangements within the University, the science 

education department and the Project frequently shared the time of staff 

members. It was not always clear that this "sharing" sufficiently freed 

faculty from former responsibilities. 

4'. The Project needs to draw upon a broader base for* its instructional 
efforts. 

. If the Project wants to teach in an interdisciplinary manner, it must 

function in an interdisciplinary mode, building appropriate contacts within the 
*\ \ • 

University. Almost the entire teaching load was borne by graduate students, 

adjuncts, and Associate Research Scientists^ Essentially, three people taught 
twenty-one of the twenty-four credits in the program - science content, educa- 
tional methodology, curriculum and field supervision. The Project staff simply 
seemed to be- trying to do too much by themselves. Involvement by other departments 
could provide new perspective, different expertise and perhaps a new insight into 
persistent problems. 

.5. The Project needs to direct its efforts to the specific curriculum 
.of New York City. • 

The Project was not designed to write curriculum and yet* the. pre-setVice 

interns were frequently involved in designingihands-on units of activities that 

were not specifically related to the New York City curriculum/ Like it or not; 

'the teachers in the City's junior high schools are committed to a specific 

curriculum. The Project needs to do a better job of teaching its trainees how 

to apply its instructional methodology to a given curriculum. 

B. jRecommendatioris for th6 Model District Program 

1. The Project should attempt to make better use of personnel within 
• the New York City School Districts. 
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The New York City school system is staffed by some capable, highly 

sophisticated people; The system has, as one would expect, an inertia and 

a certain penchant for maintainance of procedures to which it has grown 

accustomed.* These drawbacks, however, are not by any means the sum total of 

what the system Has to offer. If only the impediments' are seen, then the sole 

approach conceived of will, he hoy to minimize the blockages they % represent , 

♦ * 

That would lead to a strategy of avoidance, i.e. the less contact with certain 

forces the greater* the likelihood that Project .-efforts would be impeded. 

We believe that something like that occured with PCS and it circumscribed progress. 

2. The Project staff should state more clearly the changes *they are 
seeking to bring about. 

Any attempt, to introduce a change in'to # schools means one must_ deal with some 



existing behavioral or programmatic regularities. The project should be capable of 
stating those regularities tftat exist and noting the changes which it intends to 
bring about in them. ^ ... 

3.- The Project Should strive to me6t the. original, goals of t:he Model 
District Program. . 

To accomplish this recommendation, these goals must generate- an appropriately - 
implemented program. Appropriate implementation is not ideal ^implementation but 
rather enactment to the point where the* essential elements of the ^original 
innovation are left intact. * 4 A 

S ' 

.4, The Project should conduct its workshops in the school districts. 

4 

.Ihts was^a common suggestion for improving the competence of the cooperating 
teachers. The Project has done this in the past but gradually the workshops 
appear to have been relocated at the University. Such , a tendency is not un- 
expected £ut would appear to defeat; the purpose of*the workshops by making them 

* § • 

less available, in practical terms, to teachers. 
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C. Recommendations for the Dissemination Program 

1. The dissemination effort must move to the level of. describing ^ 
' ' research and offering, data about the, results of its training 

model* 

H 

To be an effective long range spokesman for science education in urban ^ 
centers, PCS must design a dissemination plan that goes well beyond sharing 
information which only describes activities and g&als. Up to this time much 
of the dissemination phase has dealt with recruitment of students, publicity to 
districts, and program information to other cities and institutions of higher 
education. After five years the Project should begin to show results f negative 
or positive, so that other innovative science education' efforts can replicate 
the PCS model. A serious -lack of reporting mechanisms and an absence of 
feedback data on PCS drop-outs, program graduates, and its experimental schools, 
hindered the dissemination of the model to other areas of the City. 

2. The Project staff need to reconsider and clarify their choice 
of an educational change model. 

Project personnel need to more precisely determine what concept of 

educational change they believe in and want tojuse for the remainder of the 

Project 1 s existence. It is also recommended that PCS consult with outside 

Change specialists with the intent of setting clear directions fdr future 

dissemination activites. The sine qua non for effective dissemination is 

effective Project implementation. That requires a carefully conceptualized 

modBl for changing teacher and student attitudes in, the3e experimental schools. 

. ' m _ 

3. Dissemination must be made in format that insures fidelity to 
the PCS model/ V 

The PCS struggle ttf maintain the Project and to establish new funding 

sources may increase the danger that there will be further loss of Project 

identity. Anyprecipitous or dramatic changes In the form and substance of the 

Project could leave only a skeleton of what was intended or accomplished. 
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If the Project staff truly feels thit it has a solid preservice model, and 
that they can shape model districts or .initiate 'valuable field studies, then 
they mist not stray too far from the original approach. If the staff manipulates 
and recasts the role of on-site coordinators by selecting them in ways that fit 
individual school districts but not those of the Project, PCS may end up with 
juat, another student teacher program. • * 

'* < 

■ 4. The Project should attempt to define its intended audience more 
clearly. < 

The PCS staff must consider whether or not the evidence of implementation 

results over five years supports the assumption that the Project has equally 

positive advantages for all urban junior high schools. This recommendation 

is intended to suggest* that a definite limitation of the audience targeted 

^ for PCS dissemination be considered. 

D. Recommendations for the Research Program * ^ 

t 

1. A broader coalition of those with related interests in research 
should be sought. 

-The research expertise of other educational programs such as educational 
sociology, educational psychology, and tj^e resident bureaus such as the Office 
** «of Institutional Research could have been more intimately involved 'in the 

planning 4 of research and evaluation projects. The sense of the effort conducted 
is that the small cadre -of professionals in the Project who were seriously 
pursuing ideas felt somewhat alone in their pursuit of relevant research and 
evaluation projects which would have measurably contributed to the Project s 

a 

success. It is urged that greater efforts be made to interest outside groups 
and demonstrate the opportunities that PCS presents to them* 

2. Professional help from outside the Project could have been fruitfully 
used. 
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This Prcuject did -not lack researchable areas. It suffered pr^arily from 
being unable to translate these topics' into research and evaluation design. At 
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a relatively small cost to the project, knowledgeable persons in science ^education 
research could ha^been consulted with Pr6ject personnel on a regular basis ♦ 
That remains possible. The skill the staff must demonstrate is that of attracting 
sufficient/interest in its efforts. Some .of that can be accomplished through 
trie conduct of research that draws attention to 'the unique opportunities such a 
Project provides* Other interfest can be attracted through dissemination efforts. 

3. -More attention should haye been paid to examining the relationship 

pf the research- conducted to the primary needs and interests of the ^ 
Project. 

The fact that the studies and analysis do not neatly fit into "science 
education" does not make them without merit. The efforts may ultimately be a 
contribution* to the overall attempt to understand variables which affect the 
learning of junior high school students. The studies, however, have not maximally 
informed the Project about how well they are doing nor "how they could improve 
their efforts. Few of the studies completed can be regarded as providing 
information to the Project staff regarding the success of their teaching, 
curriculum development, administration, recruitment, placement, and community 
liaison activities. 

4. The planning for the funding, staffing and activities of the Research 
Institute should have proceeded concurrently with all other ?roject 
activities. 

If this institute was as important as it appeared to be in the original 
proposals, it should have received far more attention than it did during (he 
five year term of the Project. Perhaps the original conception of an Institute 
was much too extensive for what could be accomplished in this Project. Based on 
Proj-ect documents, it was conceived to be an autonomous center ultimately' 
separated from the Project, supported independently, and evolving from five 
years of experience in research and evaluation of science education. It now 
seems evident even to the Project staff that such a goal was extremely optimistic. 
An\ Institute with capabilities for dissemination of knowledge and conduct of 
basic research with a national scope has not been developed. Of perhaps equal N 
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signigicance,' planning for the Project's Phase III continuation does not -appear 
to include a serious effort in the area of research and evaluation. < 
E. ' Policy Recommendations 

1. The NSF should reconsider and clarify its poiicy regarding Projects 
repqrts on progress. , . ' b • n 

The quarterly. (revised to trimester) reports required of the Project 

* c 

appeared excessive. They drained staff resources and their very frequency 
precluded the long terra reflection that would haye made them useful;* The* 

evaluators suggest that an annual or bi-annual report would s,erve the intended - 

> I 

purpose. Additionally/ reporting which simply records in a general ly, the- various 

activities in which a project engages, should be discouraged. * kepprts should 

instead be viewed as the means for formally reporting the specific results o£ 

project activity. Interested educators might thus bs^encouraged to consult such'-. 

reports for data on successful classroom imjliementation, teacher training 

practices/or current reseatch, • 

2\ The NSF should attempt to insure that program revision's are -reasonably 
* - proportionate to budget red\ictions'wtien fc cutbacks take place. • « 

1 ' * • • • 

" * When there is a drastic, revisit in the level of funding offered a project, 

. ' ' * ' \ ' " * 

care should be exercised to assure tj&fc appropriate adjustments, have been made 

in the.^cope of what is being; a^temjSte'd.^ The development of ' such a policy 

seems particularly hecessary vhen" laVgtf scale funding is being considered but 

would apply equally t^any' circumstance c in {jrhich Sxteusive^ reductions of budget 

are involved. .DeOelofcittg. proposals is a difficult process and accepting cuts 

a. disappointment.' SThtfse involved are unlikely to make even necessary revisions 

. V ' • * . 

at such a time J unle^s^encouraged. 'As we have indicated elsewhere, once 

♦ f\y . ' ; ■ t \ 

projects 1 ^ccrae *o^^rational ;tli^re ntfay no longer b # e time to properly plan such # m 

revisions. ' . • • , 

* w f 



3. The NSF should Assure deeper levels of commitment and support for 
•staff involvement from university officials. 

The support anticipated from the wider university community should be 

specif ied »in projects such as PCS. Too often, university officials see no 

commitment beyond that, of the particular staff for whom the funds are provided. ~' 

Broader support is needed, *and should be expected, if the university truly 

belfeves in the program that its staff has developed* > tt should not be con- 

sidered normal for such programs to cease when funding is concludes. - # . 

4. - A greater effort: should be made to insure that prefects requiring .schoot= 

university cooperation are not dominated by the needs and perspective * . 
of the university. • J * 

* • 
Th^ sahools should be brought into a full and realistic partnership, one 

in which they actively support the major purposes of the program. The schools 

must view themselvfes as co-sponsors, not as passive onlookers offering ttfeir ; 

facilities'^ exchange for <some additional help over which theyhave little 

control and limited interest. The latter has been the more common "partnership" 

that colleges have established with th& schools.' A truly joint effort would 

r^p|^e that the ^university relinquish some of Its powe*'.« Such a partnership 

would mean that the univeVsi^y staff .would actually invite the schools to examine 

the. ideas they are seeking to implement, with the right to accept, modify or 

* « * 

reject them. Where differences of opinion on any aspect: of the program occur, 

> ' J 

true partners' would have to find .means of Resolving them. The university should 
no* longer have the luxury of^resenting a |ully developed program on a -"take it 
or Hbve,~i£" b^sis, where the schools must accept the arrabgement proposed* 
virtually intact^ or risk losing what* to them are necessary and helpful additional 

services. - 4 ' v 

" Moreoyer, steps should be taken which encourage designs that insure tha^-^ 
direct involvement of university staff in the schools. Too much emphasis has „ ^ 



been placed on' models in which the practices advocated by the University are 

^demonstrated by modestly trained, or inexperienced intermediaries. Continuing 

to support such -approaches allows the ideas to remain virtually untested;- for 

f v . 

it *is unlikely that novice instructors can demonstrate them adequately. If 

. university staff believe their approaphes can revitalize instruction, they 

must accept the challenge of using~ more direct means of demonstrating them. 

5. ' Catfe should be exercised when a project! requires" numerous additions 

to the regular staf^ in order to meet program objectives. 

As was suggested earlier, th'e evaluators believe that considerable risks 
are involved when key, staff positions must be filled by individuals vjjjpse v 
^capacities to perform the function are unknown. The larger the number of such 
unfilled positions, the greater is the risk involved. 

6. For' evaluations to be fully useful to project personnel, they should 
be initialed; early in the project's existence. 

*s , _ . 

. The present evaluation did not produce a report for the PCS staff to 

consider'until the end of the Project 1 s second year of refunding (and fourth 

year* of 'existence) . That was too late to.be of practical use. By then the 

project 'had settled into an operational pattern that was difficult to break, 

even when the staff agreed with the criticisms raised. 

# • 

7. The NSF ,£h'ouid fund an effort to identify successful ongbing programs o 
* 1 science education, * < 

Success jnodel identification and the support of studies seeking to 

determine their cause would appear to be a good investment. While we question ' 

t the extent to which any program - is fcrans pot table apart from -the personnel 

involved, much can be learned which is instructive and some elements can be 

adapted for use in other programs.* * 

. Inr*a similar vein, we advocate the support of quick and inexpensive 

evaluations* of projects such as that conducted by CIRCE for Project City Science 
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They. have their uses and while their brevity will generally not allow the 

; collection of a broad ran^e of data, the value of *such studies should not be- 

underestimated. Beneath the mountains of data that sometimes obscure rather #< 

than clarify, the views of skilled and perceptive observers can often provide 

M the most meaningful insights available. 

8. Projects should % be encouraged to have their major elements planned 
and in place before program operation begins. 

* 

The evaluators have been struck by the capacity of event£» to outstrip 
planning and' organization. The problem is widespread, affecting governmental 
as well as other institutions. In this project, *a momentum was developed by 
the daily operation of the program that, made its own demands and left very 
iittle time for reflection. It' is clear that if projects such as this one are 
to exercise some control over events and not be dominated Jby them, key 
organizational and "staffing elements will need to have been carefully considered 

9 

prior to the initiation of program activities. 



( 
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